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(54) DATA PROCESSOR AND DATA RECORDED MEDIUM 

(57) As shown in figure 1 , the present invention is 
provided with a data obtaining unit 1 1 1 which receives a 
network signal Sn on a network N and obtains distrib- 
uted audio data Dau in which copyright information or 
the like is embedded, and a control unit 115 which 
obtains Information indicating a watermarking method 
that has been used when embedding the copyright 
information or the like. The embedded information is 
extracted by an appropriate watermarking method on 
the basis of the used watermarking information which 
has been obtained, and the distributed audio data Dau 
in which the extracted information is embedded is 
recorded on a recording medium. 

Therefore, regardless of the watermarking method 
used for embedding the information in the distributed 
audio data Dau supplied through the network N, repro- 
duction of the distributed audio data Dau can be appro- 
priately performed according to the embedded 
information such as the copyright information, by using 
a reproduction apparatus which is a domestic electric 
equipment. 
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Description 

TECHNICAL FIELD 

5 [0001] The present invention relates to data processing apparatuses and data recording media and, more particu- 
larly, to a process of embedding a watermark (information to be embedded) in digital data such as audio data obtained 
by electronic distribution or from recording media, by using a specific watermarking method such as a predetermined 
watermarking method which is employed for versatile data reproduction apparatuses. 

w BACKGROUND ART 

[0002] In recent years, with the progress of digitization of video data and audio data, the facility of forming a perfect 
copy of original digital data has become a great problem. 

[0003] As a countermeasure against such problem, a watermarking technique has been employed. The water- 
75 marking technique is described in detail in Japanese Published Patent Applications Nos. Hei.9-191394 and Hei.9- 
191395. 

[0004] The watermarking technique is not a direct method for preventing original digital data to be reproduced, such 
as video data or audio data, from being copied, but a method for restricting copying of the original digital data by embed- 
ding copyright information relating to the copyright for the digital data, in the digital data. 

20 [0005] For example, by embedding, as a watermark, copyright information indicating the name of the copyright 
holder (information source) in digital data such as video data and audio data, the copyright information (watermark) is 
also copied when the digital data is copied. Thereby, when the copyright holder has found digital data such as video 
data or audio data formed by illegal copying, the copyright holder can prove the illegality by the copyright information 
embedded in the digital data. 

25 [0006] Hereinafter, the watermarking technique will be described specifically. 

[0007] Figures 1 8(a) and 1 8(b) are diagrams for explaining, as an example of a watermarking technique for video, 
a process of embedding information relating to the copyright holder of digital video data (relevant information Dwmi) in 
this digital video data which is to be supplied from the distributor to the user (supply digital data). Hereinafter, this rele- 
vant information is called "watermsirk" or "embedded information". 

30 [0008] To simpJifv. Jho H^riDtiov? : as )T <?hpvyn _ in;fiaLJre ; .1.?f^), J* assumed that an image F is a gray-scale still pic- 
ture, and its size $ 'each of pixels P constituting the image F has an 
integral value within a range TrorffO to 9^ f : as3 lumihance level (pixel value) indicating its brightness. 
[0009] In the watermark embedding process, initially, the image F is divided into a plurality of blocks B each com- 
prising a predetermined number of pixels (10 pixels in length x 10 pixels in width). That is, the original still picture F is 

35 divided into 25 blocks (5 blocks in length x 5 blocks in width) as shown in figure 1 8(b). At this time, the pixel values con- 
stituting image data corresponding to each block (sub-image) B are represented as the values of the respective com- 
ponents of a matrix having 1 0 rows x 1 0 columns. 

[0010] Next, this matrix is subjected to discrete cosine transform (DCT) to obtain a transformed matrix of 1 0 rows 
x 10 columns. The numerical values placed at the lower-right portion of the transformed matrix are values representing 

40 the high-frequency components of the image data corresponding to the original image. 

[001 1 ] By the way, it is known that the human sense of sight is less responsive to a change of high-frequency com- 
ponents of image data than to a change of low-frequency components thereof. In other words, even if the original image 
is subjected to image processing for changing the values of the high-frequency components to slightly different values, 
it is very difficult for the human sense of sight to recognize the difference between the processed image (reconstructed 

45 image) and the original image. The reconstructed image is, to be specific, an image corresponding to image data recon- 
structed from an inversely-transformed matrix which is obtained by subjecting the transformed matrix to inverse DCT. 
[0012] The watermarking technique for video utilizes the above-described characteristics of human sight, and it is 
able to embed information in the image data corresponding to the original image, by controlling the high-frequency com- 
ponents of the original image, without making the viewer sense degradation of image quality. 

so [0013] Although the watermarking technique for video has been described above, there is a watermarking tech- 
nique for audio. Further, there is a watermarking technique which is able to extract or remove information from video 
data or audio data in which the information is embedded. 

[0014] Moreover, according to the watermarking technique, when digital data recorded in a digital recording 
medium is copied to another digital recording medium, information embedded in the digital data can be copied as well. 
55 Further, when the digital data recorded in the digital recording medium is once converted to analog data by a DA con- 
verter and then recorded in an analog recording medium, the embedded information can be held in the analog data 
recorded in the analog recording medium. 

[0015] Hereinafter, a description will be given of the watermarking technique for audio. 
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[0016] Figure 19 is a diagram conceptually illustrating an information embedding process and an information 
extracting process for digital audio data. 

[0017] Initially, a process Pad is performed in which signature data (information to be embedded) Dwmi is embed- 
ded in digital audio data recorded as an audio data file ODau, and then the digital audio data in which the signature data 

5 is embedded is recorded as a signature-data-embedded audio data file SDau. 

[0018] Then, a process Pex is performed in which the signature data Dwmx embedded in the digital audio data is 
extracted, on the basis of the digital audio data which is recorded as the audio data file ODau and the digital audio data 
which is recorded as the signature-data-embedded audio data file SDau. 
[0019] Figure 20 is a flowchart of the information embedding process. 

10 [0020] Initially, digital audio data is subjected to blocking (step S1 ). This process is to divide the digital audio data 
into a plurality of data groups (blocks) each comprising a predetermined number of sampling data, for convenience in 
the subsequent process. 

[0021] Next, each block is subjected to Fourier transform (step S2). The arithmetic operation for Fourier transform 
will be later described in detail. 
is [0022] Thereafter, the following data transform is carried out as the watermark (information to be embedded) 
embedding process. 

[0023] The watermark is composed of multiple bits of digital data (signature data), and each bit of the signature 
data corresponds to each block. 

[0024] Initially, it is confirmed whether the value of each bit as a component of a block string (bit string) of the sig- 
20 nature data is "0" or "1" (step S3). A block corresponding to a bit of "0" is not subjected to the watermark embedding 

process. A block corresponding to a bit of "1 ' is subjected to the watermark embedding process, wherein an imaginary 

pan and a real part of a function which has been obtained by subjecting the audio data corresponding to this block to 

Fourier transform (hereinafter, referred to as a Fourier transformed function) are replaced with each other, and the real 

part is multiplied with -1 (step S4). This process is performed for each block. 
25 [0025] Then, the Fourier transformed function corresponding to each block is subjected to inverse Fourier transform 

(step S5). Thereby, audio data of each block is restored. Inverse Fourier transform will be later described in more detail. 

[0026] Through the above-described processes, information to which a normal human ear does not respond is 

embedded in the audio data. 

[0027] Hereinafter, the respective processes of the above-described watermarking technique will be described in 
30 more detail. 

[0028] Initially, Fourier transform and inverse Fourier transform will be briefly described. Fourier transform 
employed in the process of embedding a watermark (information to be embedded) is called "discrete Fourier transform" 
which is defined as follows. 

[0029] When a discrete one-dimensional real number function f(n) (neZ,0sn<N) is given, a function obtained by 
35 subjecting this f(n) to discrete Fourier transform is defined by a discrete one-dimensional complex number function F(k) 
(keZ,0ssk<N) which is given by formula (1). 

[0030] Here, Z denotes the set of whole integers. Further, formula (1 ) satisfies the conditions given by formulae (2) 
and (3). 

40 /V-1 

F{k)= £ f(n)IV w '*" (If- 0.1. • • -W-1) (1) 

n=0 

45 j 2 = -1 (2) 

W N = e J2jc/N = cos(2tc//V) + / sin{2n/N) (3) 

[0031] Further, inverse discrete Fourier transform will be described hereinafter. 
so [0032] When a discrete one-dimensional real number function f(n) (neZ,0^n<N) is given and a discrete one-dimen- 
sional complex number function F(k) (keZ,0^k<N) is a function obtained by performing discrete Fourier transform on 
the f(n), the following formula (4) holds. 

[0033] Here, Z denotes the set of whole integers. Further, formula (4) satisfies the conditions given by formulae (5) 
and (6). 



55 



AM 

'(")=£ F{n)W N kn (fr = 0,1.. • -N-1) (4) 

n-0 
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/ 2 = -i (5) 

W„ = e^ ,N = cos(2n/N) + / sin(2n/N) (6) 

5 [0034] Next, the watermark embedding process for audio data will be described more specifically. 
[0035] First of all, blocking of audio data will be described with reference to figure 21 . 

[0036] Blocking is a process to represent sample values Sound(i) of digital audio data in which a watermark is to 
be embedded (hereinafter, referred to as target audio data) as a set of blocks each comprising samples as many as the 
n-th power of 2 (2 n ). Here, it is assumed that the total number of blocks obtained by blocking the target audio data is 
io (t+1), the first block is block B 0 , the k-tb block (k: arbitrary number) is block B k , and the last block is block B t . Further, 
the sample values of the k-th block are represented by B k (j). 

[0037] The relationship between the sample values Sound(i) of the target audio data and the respective sample val- 
ues B k (j) in the block is represented by the following formula (7). 

15 B k (j) = Sound(i) 

where Z denotes the set of whole integers, k and j satisfy k,j€ Z, and i satisfies i=2 n k+j(0sj<2 n ) . 
[0038] It is needless to say that the variables n and k used here are different from the variables n and k used in for- 
mula (1) which defines general discrete one-dimensional Fourier transform and formula (4) which defines discrete one- 
20 dimensional inverse Fourier transform. 

[0039] Next, the watermark embedding process will be described. 

[0040] Initially, the audio data (sample values) B k (j) of the k-th block B k are subjected to discrete Fourier transform 
to obtain data F k (m). Here, k is a variable indicating an arbitrary block amongst the blocks B 0 ~B t , and it satisfies ke Z, 
ke [0,t (total number of blocks)]. 

25 [0041] Further, a data bit string to be inserted Is defined by a one-dimensional discrete integral number function 
U(d), and data which is obtained by embedding information in the data F k (m)(me Z,me[1 ,21) according to the value of 
each bit in the data bit string defined by the function U(d), is represented by F' k (m). 

[0042] Here, d and d n satisfy the condition (d,d n eZ). When d n satisfies d n <2 n \ U(d) is 1 or 0. When d does not 
satisfy de [1 ,d n ]), U(d) is 0. 

30 [0043] Then, F' k (m) is represented by the following formulae (8)~(1 5), wherein m satisfies me Z, me [1 ,2 n ]. 

Re(F' k (m)) = -\m(F k (m)) (when U(m)= 1) (8) 
Re(P k (m)) = Re(F k (m)) (when U(m) = 0) (9) 

35 

\jm(r k (m)) = Re(F k (m)) (when U(m) = 1) (10) 
lm(F k (m)) = \rr\(F k (m)) ( when U(m) = 0) (11) 
40 Re(P^ (2 n - m + 1)) = -\m(F k (m)) (when U(m)= 1) (12) 

Re(P fr (2 n - m + 1))= Re(F k (m)) (when U(m) = 0) (13) 
lm(P A (2 n - m + 1))= Re(F k (m)) (when U(m) = 1) (14) 
\m(F' k (2 n - m + 1)) = lm(F^(m)) (when U(m) = 0) (15) 



45 



[0044] The above -described formulae (8)-(11) are applied to the low-frequency components amongst the 2 n 
pieces of data (frequency components) F k (m) obtained by subjecting the 2 n pieces of data (sample values) B k (j) to dis- 
50 crete Fourier transform. On the other hand, the above-described formulae (12)~(15) are applied to the high-frequency 
components of the 2 n pieces of data (frequency components) F k (m) obtained by subjecting the 2 n pieces of data (sam- 
ple values) B k (j) to discrete Fourier transform. 

[0045] Further, as represented by formulae (9), (1 1 ), (1 3) and (15), a block corresponding to a bit of 0 in the signa- 
ture data bit string is not subjected to the watermark embedding process. On the other hand, as represented by formu- 
55 lae (8), (10), (12) and (14), a block corresponding to a bit of 1 in the signature data bit string is subjected to the 
watermark embedding process, in which the imaginary part and the real part of the data F k (m) obtained by Fourier 
transform on the audio data (sample values) B k (j) corresponding to this block are replaced with each other, and the real 
part is multiplied with -1 . 
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[0046] Further, the watermark embedding process is performed on pairs of the Fourier transformed data on the 
low-frequency side and the corresponding Fourier transformed data on the high-frequency side so that the target audio 
data in which information is embedded is not offensive to the ear of the listener. Here, the m-th Fourier transformed data 
F' k (m) which has been subjected to the watermark embedding process corresponds to the (2 n -m+1)th Fourier trans- 
5 formed data F' k (2 n -m+1 ) which has been subjected to the watermark embedding process. 
[0047] Next, the watermark extraction process will be described. 
[0048] Figure 22 is a flowchart of the watermark extraction process. 

[0049] Initially, audio data which has been subjected to the information embedding process is divided into plural 
blocks (step S11a), and each block is subjected to Fourier transform (step S12a). Further, audio data which has not 
10 been subjected to the information embedding process is divided into plural blocks (step S1 1b), and the audio data of 
each block is subjected to Fourier transform (step S12b). 

[0050] Then, the data obtained as the results of the above -described Fourier transform steps are compared, block 
by block, between the blocks of the audio data which has been subjected to the Information embedding process and 
the corresponding blocks of the audio data which has not been subjected to the information embedding process (step 
15 S13). 

[0051] As the result of the comparison, when the data of the corresponding blocks are equal to each other, it is 
decided that no information is embedded in the block which has been subjected to the information embedding process, 
and the signature data bit is 0 (step S1 4). When the data of the corresponding blocks are different from each other, it is 
decided that information is embedded in the block which has been subjected to the information embedding process, 
20 and the signature data bit is 1 (step S15). 

[0052] This process is repeated block by block to extract the bit string (embedded information) constituting the sig- 
nature data. 

[0053] Next, the watermark (information to be embedded) embedding process and the watermark (embedded infor- 
mation) extracting process will be described more specifically. 

25 [0054] Initially, the process of embedding information in each block will be described. 

[0055] In the following description, Sound(n) represents audio data (sample values) in one block In which signature 
data is to be embedded, and syomei[u] represents a signature data bit string to be embedded in data which is obtained 
by subjecting the audio data Sound(n) in one block to Fourier transform. Further, F(Sound](p) represents data obtained 
by subjecting the target audio data Sound(n) to discrete Fourier transform, and F[Sound](p) represents data obtained 

30 by embedding the signature data bit string in the F[Sound](p). ■ : - % 

[0056] Here, the audio data Sound(n) is a function defined ip e" •fftdgra; - sj^i^ pi • d iav^g an integer as its value, 
and n = 0,1 N. ' " 

[0057] Further, the signature data bit string syomei[u] is also a function defined in the integral space (refer to for- 
mula (16)) and having only 0 or 1 as its value, and u = 0,1. 

35 

syomei[u] = {1 ,0} (16) 

[0058] When the audio data Sound(n) is subjected to Fourier transform, the corresponding Fourier transformed 
data F[Sound](p) is obtained as follows. 

40 

N 

F[Sound](p) = £ Sound(n)e i2npn/N (17) 
n=0 

45 

[0059] This F[Sound](p) is a function defined in the integral space and having a complex number as its value, and 
p = 0,1,...N. 

[0060] Assuming that the real part of the Fourier transformed data F[Sound](p) which is a complex number is 
Re{F[Sound](p)} while the imaginary part thereof is lm{F[Sound](p)}, the data F'[Sound](p) can be represented by using 

so the above-described formulae (8)~(15) in accordance with the value of the signature data bit string syomei[u]. 

[0061 ] Assuming that the signature data bit string to be embedded in the Fourier transformed data F[Sound](p) cor- 
responding to one block is syomei[0]=1 , the first bit value F[Sound](1) of the Fourier transformed data F[Sound](p) and 
the N-th bit value F[Sound](N) thereof are subjected to the information embedding process by using the above- 
described formulae (8), (10), (12) and (14). 

55 [0062] The following formulae (18)~(21) represent the Fourier transformed data F'[Sound](1) and P[Sound](N) 
obtained in the information embedding process. 

Re{R[Sound]V)) = -\m{F[Sound)^)} (18) 
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lm{P[Sound](1)} = Re{F[Souf)d](1)} (19) 

Re{F'[Sound](N)} = -\m{F[Sound]{N)} (20) 

lm{ r[Sound]{N)} = Re{F[Sound](A/)} (21) 

where Re and im indicate the real part and the imaginary part of the complex number in { }, respectively. 
[0063] On the other hand, assuming that the signature data bit string to be embedded in the audio data Sound(n) 
corresponding to one block is syomei[1]=0, the second bit value F[Sound](2) of the Fourier transformed data 
F[Sound](p) and the (N-1)th bit value F[Sound](N-1) thereof are subjected to the information embedding process by 
using the above-described formulae (9), (1 1), (13) and (15). 

[0064] The following formulae (22)~(25) represent the Fourier transformed data F*[Sound](2) and F[Sound](N-1 ) 
obtained in the information embedding process. 

Re{F'[Sot/nd](2)} = Re{F[Sour?d](2)} (22) 

\m{F[Sound](2)} = \n\{F[Sound](2)} (23) 

Re{F'[Sound](N -1)} = Ke{F[Sound](N -1)} (24) 

lm{F[Sound](/V -1)} = \m{F[Sound]{N -1)} (25) 

[0065] By performing inverse discrete Fourier transform on the data F'[Sound](p) which has been obtained by sub- 
jecting the Fourier transformed data F[Sound](p) corresponding to the audio data Sound(n) in one block to the Informa- 
tion embedding process by using the above-described formulae (8)~(15), information-embedded audio data Sound'(n) 
is obtained as represented by the following formula (26). 

N 

Sound'in)^ £ F\Sound](p)e i2ltpn/N (26) 

■ I 

[0066] Next, the embedded-information extraction process will be described briefly. 

[0067] In this process, the Sound(n) and the Sound'(n) are respectively subjected to Fourier transform, and the 

respective Fourier-transformed data are compared. When the values of these data are different from each other, the 

signature data bit string is extracted with the signature bit data being 1 . When the values of these data are identical, the 

signature data bit string is extracted with the signature bit data being 0. 

[0068] The algorithm will be briefly described in the following. 

[0069] In { }, n moves from 1 to N in order. 

{ 

If F[Sound](n) = F[Sound'](n) does not hold, syomei[n-1]=1 
If F[Sound](n) = F[Sound'](n) holds, syomei[n-1]=0 

} 

[0070] As described above, in recent years, as a method for protecting the copyright for video data or audio data, 
the watermarking technique for embedding information indicating the copyright holder or the like (relevant information) 
in these data to clarify the data source, has been put to practical use. 

[0071] However, there are various watermarking methods for embedding such relevant information in digital data 
and, under the existing circumstances, it is difficult to handle the digital data in which the copyright information is 
embedded, by a reproduction apparatus or the like which employs a predetermined watermarking method. 
[0072] Meanwhile, an electronic distribution system has been proposed, in which video data or audio data is dis- 
tributed not through a distribution medium (data recording medium) but through a network. 

[0073] In the electronic distribution system, distribution of video data has a considerable technical problem because 
the quantity of video data is large. However, distribution of audio data can be easily realized even in the existing limited 
network band because the quantity of audio data is smaller than the quantity of video data and, therefore, the audio 
data distribution will be implemented soon. 

[0074] Hence, there will be a demand for a system for embedding relevant information which specifies the copyright 
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holder or the like in audio data to be distributed, by using a watermarking method, to identify the source of the audio 
data. 

[0075] The present invention is made to solve the above-described problems and has for its object to provide a data 
processing apparatus which can appropriately perform electronic distribution of audio data and, moreover, which can 
5 protect the copyright for the electronic-distributed audio data. 

[0076] It is another object of the present invention to provide a data processing apparatus which can convert digital 
data in which information has been embedded by using various watermarking methods, to data which can be proc- 
essed by a predetermined watermarking method. 

[0077] It is still another object of the present invention to provide a data recording medium which contains digital 
io data having a data structure by which appropriate electronic distribution of audio data is performed and the copyright 
for the electronic-distributed audio data is protected. 

[0078] It is a further object of the present invention to provide a data recording medium which contains, as digital 
data in which information has been embedded by various watermarking methods, digital data having a data structure 
which can be processed by a predetermined watermarking method. 

75 

DISCLOSURE OF THE INVENTION 

[0079] A data processing apparatus according to the present invention (Claim 1 ) is a data processing apparatus for 
receiving various kinds of information-embedded digital data which correspond to different watermarking methods and 

20 have been obtained by embedding relevant information relating to supply digital data in the supply digital data to be sup- 
plied from the distribution end to the user, and processing these information-embedded digital data. This apparatus 
comprises: data obtaining means for obtaining a desired information-embedded digital data as input digital data; 
method information obtaining means for obtaining used method information which corresponds to the input digital data 
and indicates the watermarking method used for the process of embedding the relevant information; information extrac- 

25 tion means for extracting the relevant information from the input digital data by using the used watermarking method, 
on the basis of the used method information which has been obtained; and information embedding means for embed- 
ding the extracted relevant information or processed information obtained by processing the relevant information, in the 
Input digital data or in processed digital data obtained by subjecting the input digital data to a predetermined data 
processing, by using a predetermined watermarking method. 

30 [0080] According to the present invention (Claim 2), in the-data^processing apparatus-described in Claim 1 , the rel- 
evant information embedded in the information-embedded digital data is information relating to the copyright holder of 
the supply digital data. 

[0081] According to the present invention (Claim 3), in the data processing apparatus described in Claim 1, the 
information embedding means embeds the relevant information or the processed information in the input digital data by 
35 using the predetermined watermarking method, thereby generating output digital data. 

[0082] According to the present invention (Claim 4), the data processing apparatus described in Claim 3 further 
comprises a data reproduction unit for reproducing the output digital data. 

[0083] According to the present invention (Claim 5), the data processing apparatus described in Claim 3 further 
comprises a data recording unit for recording the output digital data. 

40 [0084] According to the present invention (Claim 6), the data processing apparatus described in Claim 1 further 
comprises information removal means for removing the relevant information from the input digital data on the basis of 
the used method information obtained by the method information obtaining means, thereby generating information- 
removed digital data as the processed digital data. The information embedding means embeds the relevant information 
or the processed information in the information -removed digital data by using the predetermined watermarking method, 

45 thereby generating output digital data. 

[0085] According to the present invention (Claim 7), the data processing apparatus described in Claim 6 further 
comprises a data reproduction unit for reproducing the output digital data. 

[0086] According to the present invention (Claim 8), the data processing apparatus described in Claim 6 further 
comprises a data recording unit for recording the output digital data. 

so [0087] According to the present invention (Claim 9), in the data processing apparatus described in Claim, the infor- 
. mation-embedded digital data comprises a first data portion having a fixed length and a second data portion having a 
variable length; the used method information indicating the used watermarking method is embedded in the first data 
portion by the used watermarking method; and the method information obtaining means obtains the used method infor- 
mation indicating the used watermarking method, from the first data portion of the input digital data. 

55 [0088] According to the present invention (Claim 1 0), in the data processing apparatus described in Claim 1 , the 
informationembedded digital data has a data structure including a plurality of data packets; the used method informa- 
tion Indicating the used watermarking method is Included in a header of a predetermined data packet amongst the plu- 
ral data packets; and the method information obtaining means obtains the used method information indicating the used 
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watermarking method from the header of the predetermined data packet of the information-embedded digital data. 
[0089] According to the present invention (Claim 11), in the data processing apparatus described in claim 1, the 
information-embedded digital data is followed by an auxiliary data portion; the used method information indicating the 
used watermarking method is included in the auxiliary data portion; and the method information obtaining means 
5 obtains the used method information indicating the used watermarking method from the auxiliary data portion which fol- 
lows the information-embedded digital data. 

[0090] According to the present invention (Claim 12), in the data processing apparatus described in Claim 1, the 
data obtaining means is able to obtain the information-embedded digital data from at least one of plural data recording 
media and plural data transmission paths; and the method information obtaining unit decides the used watermarking 
10 method in accordance with any of the data recording media and the data transmission paths from which the informa- 
tion-embedded digital data has been obtained, and obtains the used method information corresponding to the result of 
the decision. 

[0091] According to the present invention (Claim 13), in the data processing apparatus described in Claim 1, the 
information-embedded digital data is audio digital data, or video digital data, or multiplexed data obtained by multiplex- 

75 ing audio digital data and video digital data. 

[0092] According to the present invention (Claim 14), in the data processing apparatus described in Claim 1, the 
data obtaining means has an antenna for receiving a broadcast wave from a broadcasting station, and a received-wave 
decoder for demodulating the wave received by the antenna to output the information-embedded digital data. 
[0093] According to the present invention (Claim 15), in the data processing apparatus described in Claim 1, the 

20 data obtaining means has a data reading unit for reading, from a bulk memory, the information-embedded digital data 
stored in the bulk memory. 

[0094] According to the present invention (Claim 1 6), in the data processing apparatus described in Claim 15, the 
data reading unit has an access unit for reading the information -embedded digital data from DVD-ROM, DVD-RAM, or 
semiconductor recording media. 
25 [0095] According to the present invention (Claim 17), in the data processing apparatus described in Claim 1, the 
information-embedded digital data is obtained by embedding plural pieces of relevant information in the supply digital 
data by using plural watermarking methods; plural pieces of used method information corresponding to the plural water- 
marking methods are included in the information-embedded digital data, as the used method information indicating the 
used watermarking method; and the information extracting means is able to extract the relevant information corre- 
- ,30 sponding to each of the plural watermarking methods from the input digital data, on the basis of the used method infor- 
mation corresponding to the plural watermarking methods. 

[0096] According to the present invention (Claim 18), in the data processing apparatus described in Claim 1, the 
information-embedded digital data is obtained by embedding plural pieces of relevant information in the supply digital 
data by using plural watermarking methods; and the information embedding means embeds a predetermined one of 
35 the plural pieces of relevant information in the input digital data by using the predetermined watermarking method, 
thereby generating output digital data. 

[0097] According to the present invention (Claim 19), in the data processing apparatus described in Claim 1, the 
information-embedded digital data is obtained by embedding plural pieces of relevant information in the supply digital 
data by using plural watermarking methods; and the information embedding means embeds a predetermined number 
40 of relevant information amongst the plural pieces of relevant information, in the input digital data, by using the corre- 
sponding watermarking methods amongst the plural watermarking methods or a watermarking method different from 
the plural watermarking methods, thereby generating output digital data. 

[0098] According to the present invention (Claim 20), in the data processing apparatus described in Claim 6, the 
information-embedded digital data is obtained by embedding plural pieces of relevant information in the supply digital 
45 data by using plural watermarking methods; and the information embedding means embeds a predetermined one of 
the plural pieces of relevant information in the processed digital data by using the predetermined watermarking method, 
thereby generating output digital data. 

[0099] According to the present invention (Claim 21), in the data processing apparatus described in Claim 6, the 
information-embedded digital data is obtained by embedding plural pieces of relevant information in the supply digital 
so data by using plural watermarking methods; and the information embedding means embeds a predetermined number 
of relevant information amongst the plural pieces of relevant information, in the processed digital data, by using the cor- 
responding watermarking methods amongst the plural watermarking methods or a watermarking method different from 
the plural watermarking method, thereby generating output digital data. 

[0100] A data recording medium according to the present invention (Claim 22) is a data recording medium in which 
55 information-embedded digital data is recorded, which information-embedded digital data has been obtained by embed- 
ding relevant information relating to supply digital data in the supply digital data to be supplied from the distribution end 
to the user. The information-embedded digital data has a data structure wherein the information-embedded digital data 
is composed of a first data portion having a fixed length and a second data portion having a variable length; used 
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method information indicating the watermarking method used for embedding the relevant information In the supply dig- 
ital data is embedded in the first data portion; the relevant information is embedded in the second data portion by the 
used watermarking method indicated by the used method information; and the relevant information can be extracted or 
removed from the second data portion by using an appropriate watermarking method on the basis of the used method 
5 information. 

[0101] According to the present invention (Claim 23), in the data recording medium described in Claim 22, the used 
method information is embedded in the first data portion of the information-embedded digital data by a predetermined 
watermarking method. 

[0102] According to the present invention (Claim 24), in the data recording medium described in Claim 22, plural 
w pieces of used method information indicating different used watermarking methods, which have been used for embed- 
ding the relevant information in the supply digital data, are embedded in the first data portion of the information-embed- 
ded digital data. 

[0103] A data recording medium according to the present invention (Claim 25) is a data recording medium in which 
information-embedded digital data and used method information are recorded, which information-embedded digital 

75 data has been obtained by embedding relevant information relating to supply digital data in the supply digital data to be 
supplied from the distribution end to the user, and which used method information indicates the watermarking method 
which has been used for embedding the relevant information in the supply digital data and follows the information- 
embedded digital data. In this medium, the information-embedded digital data has a data structure in which the relevant 
information can be extracted or removed by using an appropriate watermarking method on the basis of the used 

20 method information. 

[0104] According to the present invention (Claim 26), in the data recording medium described in Claim 25, plural 
pieces of used method information indicating different watermarking methods, which have been used for embedding 
the relevant information in the supply digital data, are recorded as the used method information. 
[0105] As described above, the data processing apparatus according to the present invention (Claim 1 ) is provided 

25 with data obtaining means for obtaining, as input digital data, information-embedded digital data which has been 
obtained by embedding relevant information relating to supply digital data in the supply digital data to be supplied from 
the distribution end to the user; method information obtaining means for obtaining used method information which cor- 
responds to the input digital data and indicates the watermarking method used for the process of embedding the rele- 
vant information; and information extraction means for extracting the relevant information from the input digital data by 

30 using the used watermarking method, on the basis of the used method information which has been. obtained The. 
extracted relevant information or processed information obtained by processing the relevant informatjij ".U;linpeoc«j4| ; 
in the input digital data or in processed digital data obtained by subjecting the input digital data to a predetermined data 
processing. Therefore, the apparatus can detect the watermarking method by which the relevant information has been 
embedded in the supply digital data. Accordingly, when extracting or removing the relevant information, it is avoided that 

35 wrong information is extracted by using another method and that a portion other than the embedded relevant informa- 
tion is undesirably changed by using another method. 

[0106] As the result, at the distribution end from which digital data such as audio data is distributed to the user, the 
relevant information can be embedded as a watermark in the supply digital data to be supplied, by using a desired 
watermarking method. In other words, especially when the digital data is to be recorded on a recording medium at the 

40 user end, the distribution end of the digital data can select a watermarking method which provides the least degradation 
due to the information embedding process, according to the content of the digital data. Further, the distribution end can 
select the optimum method according to the communication rate or the like when transmitting the digital data. 
[0107] Further, when the information-embedded digital data obtained from the network is recorded on a writable 
recording medium, a watermarking method, which is different from the watermarking method used for embedding infor- 

45 mation in the supply digital data at the distribution end, can be used for embedding information in the digital data to be 
written in the recording medium. Therefore, information can be embedded in the digital data by a watermarking method 
adapted to an apparatus which reproduces the digital data. 

[0108] According to the present invention (Claim 2), in the data processing apparatus described in Claim 1 , the rel- 
evant information embedded in the information-embedded digital data is information relating to the copyright holder of 
so the supply digital data. Therefore, the copyright for the supply digital data to be supplied from the distribution end to the 
user can be protected. 

[0109] According to the present invention (Claim 3), in the data processing apparatus described in Claim 1, the 
information embedding means embeds the relevant information or the processed information in the input digital data by 
using the predetermined watermarking method, thereby generating output digital data. Therefore, regardless of the 
55 watermarking method by which the relevant information has been embedded in the obtained digital data, reproduction 
apparatuses as domestic electrical equipment can appropriately reproduce the information -embedded digital data, 
according to the embedded information like copyright information, on the basis of the output digital data. 
[0110] According to the present invention (Claim 4), the data processing apparatus described in Claim 3 further 
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comprises a data reproduction unit for reproducing the output digital data. Therefore, reproduction of the obtained infor- 
mation-embedded digital data can be controlled according to the relevant information. 

[0111] According to the present invention (Claim 5), the data processing apparatus described in Claim 3 further 
comprises a data recording unit for recording the output digital data. Therefore, regardless of the watermarking method 
used for the information-embedded digital data, the information-embedded digital data can be stored, as digital data 
corresponding to a predetermined watermarking method, in recording media which are used in reproduction appara- 
tuses as domestic electrical equipment. 

[011 2] According to the present invention (Claim 6), in the data processing apparatus described in Claim 1 , the rel- 
evant information or the processed information is embedded in processed digital data which has been obtained by sub- 
jecting the input digital data to a predetermined data processing, by using the predetermined watermarking method. 
Therefore, the relevant information in the input digital data, which corresponds to various watermarking methods, can 
be replaced with that corresponding to the predetermined watermarking method. 

[0113] According to the present invention (Claim 7), the data processing apparatus described in Claim 6 further 
comprises a data reproduction unit for reproducing the output digital data. Therefore, reproduction of the obtained infor- 
mation-embedded digital data can be controlled according to the relevant information. Further, when the processed dig- 
ital data is digital audio data which has been obtained by removing the embedded information from the input digital 
data, reproduction of the information-embedded digital data can be performed according to the processed digital data, 
with minimized degradation of sound quality. 

[0114] According to the present invention (Claim 8), the data processing apparatus described in Claim 6 further 
comprises a data recording unit for recording the output digital data. Therefore, regardless of the watermarking method 
used for the obtained information-embedded digital data, the information-embedded digital data can be stored in 
recording media used in reproduction apparatuses as domestic electrical equipment, as digital data having less noise 
and corresponding to the predetermined watermarking method. 

[0115] According to the present invention (Claim 9), in the data processing apparatus described in Claim, the infor- 
mation-embedded digital data comprises a first data portion having a fixed length and a second data portion having a 
variable length; and the method intonation obtaining means obtains the used method information from the first data por- 
tion of the information -embedded digital data. Therefore, the method information obtaining means can easily detect the 
position of the used method information in the information-embedded digital data. 

[0116] According to the present invention (Claim 10), in the data processing apparatus described in Claim 1, the 
information-embedded digital data has a data structure including a plurality of data packets; and the method information 
obtaining means obtains the used method intonation indicating the used watermarking method from the header of a 
predetermined data packet of the information-embedded digital data. Therefore, the method information obtaining 
means can easily detect the position of the used method information in the information-embedded digital data. 
[0117] According to the present invention (Claim 11), in the data processing apparatus described in Claim 1, the 
information-embedded digital data is followed by an auxiliary data portion; and the method information obtaining means 
obtains the used method information indicating the used watermarking method from the auxiliary data portion which fol- 
lows the information-embedded digital data. Therefore, the method information obtaining means can easily detect the 
position of the used method information in the information-embedded digital data. 

[0118] According to the present invention (Claim 12), in the data processing apparatus described in Claim 1, the 
used watermarking method is decided in accordance with the path through which the information-embedded digital 
data has been obtained, and the used method information corresponding to the decision is obtained. Therefore, it is not 
necessary to insert the watermarking method information in the digital data to be distributed from the distribution end 
to the network. Further, it is not necessary to insert the watermarking method information in the distributed digital data 
to be stored in the recording medium. Thereby the data processing for the supply digital data to be supplied from the 
distribution end to the user can be reduced. 

[0119] According to the present invention (Claim 13), in the data processing apparatus described in Claim 1, the 
information-embedded digital data is audio digital data, or video digital data, or multiplexed digital data obtained by mul- 
tiplexing audio digital data and video digital data. Therefore, the relevant information embedded in the various kinds of 
digital data (e.g., audio digital data, video digital data, multiplexed digital data), which corresponds to various water- 
marking methods, can be converted to that corresponding to the predetermined watermarking method. 
[0120] According to the present invention (Claim 14), in the data processing apparatus described in Claim 1, the 
data obtaining means has an antenna for receiving a broadcast wave from a broadcasting station, and a received-wave 
decoder for demodulating the wave received by the antenna to output the information -embedded digital data. Therefore, 
information-embedded digital data can be obtained from the broadcast wave transmitted from the broadcast station. 
[0121] According to the present invention (Claims 15 and 16), in the data processing apparatus described in Claim 
1 , the data obtaining means has a data reading unit for reading the information-embedded digital data from a bulk mem- 
ory. Therefore, the information-embedded digital data can be obtained from DVD-ROM, DVD-RAM, or semiconductor 
recording media. 
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[0122] According to the present invention (Claim 17), in the data processing apparatus described in Claim 1, the 
information-embedded digital data is obtained by embedding plural pieces of relevant information in the supply digital 
data by using plural watermarking methods, and each of these relevant information is extracted by the corresponding 
watermarking method. Therefore, various kinds of relevant information can be embedded in the supply digital data by 

5 appropriate watermarking methods. 

[0123] According to the present invention (Claim 18), in the data processing apparatus described in Claim 1, the 
information-embedded digital data is obtained by embedding plural pieces of relevant information in the supply digital 
data by using plural watermarking methods, and a predetermined one of these relevant information is embedded In the 
input digital data by the predetermined watermarking method. Therefore, a desired relevant information amongst the 

io various kinds of relevant information embedded in the digital data can be adapted to the predetermined watermarking 
method. 

[0124] According to the present invention (Claim 19), in the data processing apparatus described in Claim 1, the 
information-embedded digital data is obtained by embedding plural pieces of relevant information in the supply digital 
data by using plural watermarking methods, and a predetermined number of relevant information amongst the plural 
is pieces of relevant information are embedded in the input digital data by using the corresponding watermarking methods 
amongst the plural watermarking methods or a watermarking method different from the plural watermarking methods. 
Therefore, a desired relevant information amongst the various kinds of relevant information embedded in the digital data 
can be adapted to a required watermarking method. 

[0125] According to the present invention (Claim 20), in the data processing apparatus described in Claim 6, the 
20 information-embedded digital data is obtained by embedding plural pieces of relevant information in the supply digital 
data by using plural watermarking methods, and a predetermined one of the plural pieces of relevant information is 
embedded in the processed digital data by using the predetermined watermarking method. Therefore, the information- 
embedded digital data containing various kinds of relevant information can be converted to new digital data in which 
desired one of the various kinds of relevant information is adapted to the predetermined watermarking method. 
25 [0126] According to the present invention (Claim 21), in the data processing apparatus described in Claim 6, the 
information-embedded digital data is obtained by embedding plural pieces of relevant information in the supply digital 
data by using plural watermarking methods, and a predetermined number of relevant information amongst the plural 
pieces of relevant information are embedded in the processed digital data by using the corresponding watermarking 
methods amongst the plural watermarking methods or a watermarking method different from the plural watermarking 
30 method. Therefore, the information-embedded digital data containing various kinds of relevant information can be con- 
verted to new digital data in which desired one of the various kinds of relevant information is adapted to a required 
watermarking method. 

[0127] A data recording medium according to the present invention (Claim 22) is a data recording medium In which 
information-embedded digital data is recorded, which information-embedded digital data has been obtained by embed- 

35 ding relevant information relating to supply digital data in the supply digital data to be supplied from the distribution end 
to the user. The information-embedded digital data is composed of a fixed-length first data portion in which used 
method information indicating used watermarking method is embedded, and a variable-length second data portion in 
which the relevant information is embedded by the used watermarking method. Therefore, an apparatus which obtains 
the information-embedded digital data can decide the watermarking method used for embedding the information in the 

40 digital data. Thereby, the apparatus can perform extraction or removal of the embedded information, without extracting 
wrong information by using another method when extracting the embedded information, or without changing a portion 
other than the embedded information by using another method when removing the embedded information. Further, the 
apparatus which obtains the information-embedded digital data can easily detect the position of the used method infor- 
mation in the information-embedded digital data. 

45 [0128] As the result, at the distribution end from which digital data such as audio data is distributed to the user, the 
relevant information can be embedded as a watermark in the supply digital data to be supplied, by using a desired 
watermarking method. In other words, especially when the digital data Is to be recorded on a recording medium at the 
user end, the distribution end of the digital data can select a watermarking method which provides the least degradation 
due to the information embedding process, according to the content of the digital data. Further, the distribution end can 

50 select the optimum method according to the communication rate or the like when transmitting the digital data. 

[0129] Further, when the information-embedded digital data obtained from the network is recorded on a writable 
recording medium, a watermarking method, which is different from the watermarking method used for embedding infor- 
mation in the supply digital data at the distribution end, can be used for embedding information in the digital data to be 
written in the recording medium. Therefore, information can be embedded in the digital data by a watermarking method 

55 adapted to an apparatus which reproduces the digital data. 

[0130] According to the present invention (Claim 23), in the data recording medium described in Claim 22, the used 
method information is embedded in the first data portion of the information-embedded digital data by a predetermined 
watermarking method. Therefore, extraction of the used method information indicating the used watermarking method 
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is facilitated. 

[0131] According to the present invention (Claim 24), in the data recording medium described in Claim 22, plural 
pieces of used method information indicating different used watermarking methods are embedded as the used method 
information. Therefore, an apparatus which obtains the information-embedded digital data can control reproduction of 

5 the digital data on the basis of the plural pieces of used method information. 

[0132] A data recording medium according to the present invention (Claim 25) is a data recording medium in which 
information-embedded digital data and used method information are recorded, which information-embedded digital 
data has been obtained by embedding relevant information relating to supply digital data in the supply digital data to be 
supplied from the distribution end to the user, and which used method information indicates the watermarking method 

10 which has been used for embedding the relevant information In the supply digital data and follows the information- 
embedded digital data. In this medium, the information-embedded digital data has a data structure in which the relevant 
information can be extracted or removed by using an appropriate watermarking method on the basis of the used 
method information. Therefore, an apparatus which obtains the information-embedded digital data can perform 
processing or reproduction of the digital data on the basis of the used method information. 

75 [0133] Further, like Claim 22, the apparatus can decide the watermarking method used for embedding the informa- 
tion in the digital data. Therefore, the apparatus can perform extraction or removal of the embedded information without 
extracting wrong information by using another method when extracting the embedded information, or without changing 
a portion other than the embedded information by using another method when removing the embedded information. 
[0134] According to the present invention (Claim 26), in the data recording medium described in Claim 25, plural 

20 pieces of used method information indicating different watermarking methods are embedded as the used method infor- 
mation. Therefore, an apparatus which obtains the information-embedded digital data can perform processing or repro- 
duction of the digital data on the basis of the plural pieces of used method information. 

BRIEF DESCRIPTION OF THE DRAWINGS 

25 

[0135] 

Figure 1 is a block diagram for explaining a data processing apparatus according to a first embodiment of the 
present invention. 

30 Figure 2 is a diagram illustrating the data structure of distributed audio data Dau supplied by a network signal to the 

data processing apparatus of the first embodiment. 

Figure 3 is a diagram for explaining used watermarking method information included in the distributed audio data 
Dau. 

Figure 4 is a block diagram for explaining a data processing apparatus according to a second embodiment. 
35 Figure 5 is a diagram illustrating the data structure of used watermarking method information included in audio data 

attribute information supplied to the data processing apparatus of the second embodiment. 

Figure 6 is a block diagram for explaining a data processing apparatus according to a third embodiment. 

Figure 7 is a block diagram for explaining a data processing apparatus according to a fourth embodiment. 

Figure 8 is a diagram illustrating the data structure of a video object to be processed by the data processing appa- 
40 ratus of the fourth embodiment. 

Figure 9 is a diagram illustrating the data structure of an audio packet included in the video object 

Figure 10 is a diagram illustrating the values of 2-bit data stored in the header Pah of the audio packet as used 

watermarking method information lum, and the meaning of each value. 

Figure 11 is a block diagram for explaining a data processing apparatus according to a fifth embodiment. 
45 Figure 1 2 is a diagram illustrating the data structure of distributed audio data recorded on a data recording medium, 
which data is to be processed by the data processing apparatus of the fifth embodiment. 

Figure 1 3 is a diagram illustrating the data structure of used watermarking method information lum included in TOK 
information in the distributed audio data. 

Figure 1 4 is a block diagram for explaining a data processing apparatus according to a sixth embodiment. 
so Figures 15(a) is a diagram illustrating the data structure of embedded information (first embedded information) in 
distributed audio data supplied by a broadcast wave to the data processing apparatus of the sixth embodiment, and 
figure 15(b) is a diagram illustrating the data structure of processed embedded information (second embedded 
information) obtaining by processing the first embedded information. 

Figure 1 6(a) is a diagram illustrating the values of information Irm of watermarking method used at recording which 
55 is included in the first embedded information lwmT,~and the meaning of each value, and figure* 16(b) is a diagram 
illustrating the values of information lum of watermarking method used at recording which is included in the second 
embedded Information Iwm2, and the meaning of each value. 

Figure 1 7 is a block diagram for explaining a data processing apparatus according to a seventh embodiment of the 
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present invention. 

Figures 1 8(a) and 1 8(b) are diagram for explaining the method for embedding information (watermark) relating to 
the copyright holder of digital video data, by using watermarking technique, wherein 1 8(a) shows a matrix of pixels 
constituting an image, and 1 8(b) shows a matrix of pixels in each of plural blocks obtained by dividing the Image. 
5 Figure 19 is a diagram conceptually illustrating an information embedding process and an information extraction 

process in/from digital audio data. 

Figure 20 is a flowchart of the information embedding process. 

Figure 21 is a diagram for explaining blocking of audio data in the information embedding process. 
Figure 22 is a flowchart of the information extraction process. 
io Figure 23 is a conceptual diagram illustrating the flow of digital audio data in an electronic distribution system which 

is a premise of the present invention. 

BEST MODE TO EXECUTE THE INVENTION 

15 [0136] The inventors of the present invention have earnestly studied the recent watermarking technique and the 
electronic distribution system for digital data, and discovered that, when employing the watermarking technique for the 
electronic distribution system, the variety of existing watermarking methods will be a great problem. 
[0137] To be specific, the supply of audio data by electronic distribution can be performed very easily as compared 
with the supply of audio data by data recording media, and so the number of the data sources will Increase considera- 
te bly. Therefore, it is impossible to unify the watermarking methods amongst the distributors of the respective audio data. 
On the other hand, in the electronic distribution system, it is very easy to obtain audio data and so the possibility of ille- 
gal copying is high. Therefore, to unify the watermarking methods amongst the different distributors is not desirable 
because, in this case, illegal utilization of audio data cannot be supervised when the watermarking method is 
decrypted. 

25 [0138] Hereinafter, the problem in the electronic distribution system and the countermeasure against the problem 
will be described in more detail. 

[0139] Figure 23 is a diagram illustrating the flow of digital audio data in the electronic distribution system. 
[0140] With reference to figure 23, a receiving/recording unit 200 is connected to a network N. Information-embed- 
ded digital data in which Information is embedded by various watermarking methods, exist on the network N. In figure 

30 23, three watermarking methods (method A, method B, and method C) are employed, and digital audio data Da, Db, 
and Dc in which information is embedded by the respective methods are supplied to the receiving/recording unit 200. 
Further, the receiving/recording unit 200 records the received digital data Do1 on a data recording medium Rd. This 
system further includes a reproduction unit 300 on which the data recording medium Rd is mounted, wherein the digital 
data Do2 recorded on the data recording medium Rd is reproduced. It is assumed that a personal computer is used as 

35 the receiving/recording unit 200, and a semiconductor player or a DVD player (domestic electrical equipment) is used 
as the reproduction unit 300. 

[0141] In the electronic distribution system so constructed, initially, the digital audio data Da, Db, and Dc are 
received by the receiving/recording unit 200, and the received digital audio data is output to the data recording medium 
Rd as the output digital data Dot. Then, the output digital data Do1 is recorded in the data recording medium Rd. 
40 [0142] Next, in the reproduction unit 300 on which the data recording medium Rd is mounted, the digital data Do2 
which has been read from the data recording medium Rd Is reproduced. 

[0143] By the way, the reproduction unit 300 is a domestic equipment the specification of which cannot be changed 
easily after marketing of it has started and, therefore, it is difficult to change the watermarking method employed in the 
reproduction unit 300. 

45 [0144] Accordingly, in the electronic distribution system for audio data, although it is necessary to supply digital 
audio data adapted to the predetermined watermarking method to the reproduction unit 300, it is impossible to unify the 
watermarking methods amongst the distributors of the respective digital audio data. 

[0145] In order to solve this problem, the inventors of the present invention noticed that the personal computer used 
as the receiving/recording unit 200 can be adapted to digital data which have been processed by various watermarking 

so methods, by changing the software. 

[0146] However, even if the software is changed, the receiving/recording unit 200 cannot specify the watermarking 
method by which information has been embedded in the received digital data and, therefore, the embedded information 
cannot be extracted by the watermarking method suitable for the received digital data. To be specific, the arithmetic 
process for extracting or removing the embedded information from the digital data is the inverse of the arithmetic proc- 

55 ess for embedding the information in the digital data, and the same watermarking method is used for the information 
embedding process and the information extraction process. Therefore, if the watermarking method used for the infor- 
mation embedding process cannot be specified, the Information extraction or removal process cannot be performed. 
[0147] So, the inventors of the present invention has earnestly studied the electronic distribution system and finally 
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discovered the following fact That is, in the electronic distribution system, when the receiving/recording unit 200 con- 
stituted by a personal computer is provided with the function of specifying the watermarking method used for embed- 
ding information in the received digital data, the digital data in which information has been embedded by various 
watermarking methods can be converted to data adapted to a specific watermarking method used for the reproduction 
s unit 300 as a domestic equipment. 

[0148] For example, the digital data Da, Db, and Dc recorded by the above-described three methods A, B, and C, 
respectively, can be convened to the digital data Dx adapted to the specific watermarking method used for the repro- 
duction unit 300. 

[0149] More specifically, the embedded information is extracted from the received digital data by using the same 
io watermarking method as that used for embedding the information in the digital data, and then the extracted information 
corresponding to the used watermarking method is again embedded in the received digital data by using a specific 
watermarking method employed in the reproduction unit 300. Thereby, regardless of the watermarking method used for 
embedding the information in the received digital data, the reproduction unit 300 as a domestic equipment can appro- 
priately reproduce the received digital data. 
is [0150] Hereinafter, the embodiments of the present invention will be described in detail. 

Embodiment 1 . 

[0151] Figure 1 is a block diagram for explaining a data processing apparatus 1 1 0 according to a first embodiment 
20 of the present invention. 

[0152] The data processing apparatus 110 obtains various kinds of information -embedded digital data Dau corre- 
sponding to different watermarking methods from a network signal Sn on a network N, processes each of the Informa- 
tion-embedded digital data so that it corresponds to a predetermined watermarking method, and records the digital data 
corresponding to the predetermined watermarking method in a data recording medium. 

25 [01 53] In this first embodiment, for simplification, it is assumed that the data processing apparatus 1 1 0 receives, as 
the various kinds of information-embedded digital data Dau, digital data Da, Db, and Dc which correspond to the three 
watermarking methods (method A, method B, and method C) shown in figure 23, respectively. To be specific, the digital 
data Da is distributed audio data which is obtained by embedding, as a watermark, relevant information (information to 
be embedded) in supply audio data to be supplied from the distribution end to the user, by using the watermarking 

3p method A. The digital data Db is distributed audio data which is obtained by embedding relevant information in another 
supply digital audio data by using the method B. The digital data Dc is distributed audio data which is obtained by 
embedding relevant information in still another supply digital audio data by using the method C. Further, the predeter- 
mined watermarking method is a versatile watermarking method employed for reproduction apparatuses which are 
domestic electrical equipment, such as semiconductor players or DVD players. 

35 [0154] Turning to figure 1 , the data processing apparatus 1 1 0 includes a unit (data obtaining unit) 1 1 1 for obtaining, 
as input digital data, the information-embedded digital data Dau (any of the digital data Da, Db, and Dc) from the net- 
work signal Sn on the network N. This data obtaining unit 1 1 1 comprises a network interface 111a which exchanges a 
transmission signal with the network N, and a received signal decoder 111b which converts the network signal Sn 
received through the interface 1 11a to digital data by decoding. That is, the received signal decoder 111b decodes the 

40 network signal Sn and outputs the information-embedded digital data Dau as input digital data (distributed audio data). 
[0155] Further, the data processing apparatus 110 includes a control unit 1 15 which has a function (method infor- 
mation obtaining means) for obtaining, from the input digital data, used method information indicating the watermarking 
method used for embedding the relevant information in the supply digital data; and an extraction unit 1 12 which extracts 
the embedded information (relevant information) from the input digital data, by the used watermarking method in 

45 accordance with the used method information. 

[0156] The control unit comprises a CPU which controls various kinds of arithmetic processing and data process- 
ing, and a RAM which contains the operation program of the CPU and also serves as an execution memory. Accord- 
ingly, in the control unit 115, the method information obtaining means is mounted on the CPU. Further, the extraction 
unit 112 is constituted by a custom LSI. The extraction unit 112 comprises a method A extractor 112a which extracts 

so the embedded information from the digital data Da by the watermarking method A, a method B extractor 1 12b which 
extracts the embedded information from the digital data Db by the watermarking method B, and a method C extractor 
1 12c which extracts the embedded information from the digital data Dc by the watermarking method C. 
[0157] The data processing apparatus 110 further includes an insertion unit (information embedding means) 113 
which embeds the extracted information in the input digital data (distributed audio data) by the predetermined water- 

55 marking method to generate output digital data; and a recording unit 1 14 which stores the output digital data. 

[0158] The insertion unit 1 13 is constituted by a custom LSI, and the recording unit 1 14 is provided with a DVD- 
RAM drive which writes and reads the output digital data in/from a DVD-RAM disk. 

[0159] In the data processing apparatus 110, the data obtaining unit 1 1 1 , the extraction unit 112, the insertion unit 
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113, the recording unit 114, and the control unit 1 1 5 are connected with each other by a data bus Dbus, and the control 
unit 115 controls the operations of the data obtaining unit 1 1 1 , the extraction unit 112, the insertion unit 1 13, and the 
recording unit 1 14. That is, the control unit controls, not only the operation for obtaining the used watermarking method 
information from the input digital data, but also the following processes: the process of obtaining the information- 
5 embedded digital data Dau from the network signal Sn by the data obtaining unit 111, the process of extracting the 
embedded information by the extraction unit 112, the process of embedding the extracted information by the insertion 
unit 1 13, and the process of writing the output digital data in the recording unit 1 1 4. 

[0160] In the data processing apparatus 1 1 0 of this first embodiment, the digital data obtaining unit 1 1 1 comprises 
the network interface 111a and the received signal decoder 1 1 1 b, i.e., it is constructed so as to receive the network sig- 

w nal Sn from the network N through a communication line and decode this signal to obtain the information-embedded 
digital data Dau. However, the construction of the digital data obtaining unit is not restricted thereto. 
[0161] For example, the digital data obtaining unit 111 may comprise an antenna which receives a broadcast wave, 
and a received wave decoder which demodulates the received wave output from the antenna, and the output from the 
received wave decoder may be obtained as the information-embedded digital data Dau. 

75 [0162] Further, the digital data obtaining unit may comprises a modem connected to a telephone line or a coaxial 
cable. In this case, an analog transmission signal from the telephone line or the coaxial cable is converted to informa- 
tion-embedded digital data by the modem, and this information-embedded digital data is obtained as the input digital 
data. 

[0163] Further, the digital data obtaining unit 111 may be a bulk storage which reads the information-embedded 
20 digital data Dau from bulk data storage media capable of handling multimedia data, such as disk storage media like 
DVD-ROM, DVD-RAM, and CD-ROM, or semiconductor media. 

[0164] While in this first embodiment the extraction unit 1 1 2 and the insertion unit 1 1 3 are constituted by separated 
custom LSI circuits, these units may be constituted by a single custom LSI. As the extraction units 112 and the insertion 
unit 113 perform a lot of common processes, to constitute these units by a single custom LSI is desirable for reduction 

25 in the number of components. 

[0165] Further, while in this first embodiment the extraction unit 112 and the insertion unit 1 13 are constituted by 
custom LSI circuits, these units may be constituted by a CPU performing arithmetic processing and a RAM which stores 
the arithmetic program and serves as an execution memory. Especially in this case, the extraction unit 1 1 2 and the 
insertion unit 113 can share the constituents such as the CPU and the memory with the control unit 115, whereby the 

30 number of components can be reduced effectively. 

[0166] Furthermore, while in this first embodiment the recording unit 114 is constituted by a DVD-RAM drive, this 
recording unit 114 may be constituted by any storage as long as it is a bulk writable storage capable of handling multi- 
media data, such as a CD-R drive or a semiconductor media writing unit. 

35 (data structure of digital data) 

[0167] Next, a description will be given of the data structure of the information-embedded digital data Dau which is 
obtained from the network signal Sn on the network N, and the output digital data which is output from the insertion unit 
113. 

40 [0168] These digital data have the same data structure, and figure 2 shows the data structure of the information- 
embedded digital data Dau. 

[0169] The information-embedded digital data Dau is composed of a first data section Dau1 and a second data sec- 
tion Dau2. Although the size (length) of the information-embedded digital data Dau is variable, the size of the first data 
section Dau1 is fixed. Therefore, the data processing apparatus which has received the information-embedded digital 

45 data Dau can distinguish the first data section Dau1 from the second data section Dau2. 

[0170] In the digital data corresponding to the first data section Dau1 , used method information indicating the used 
watermarking method is embedded by a versatile watermarking method amongst the above-described watermarking 
methods A, B, and C. On the other hand, in the digital data corresponding to the second data section Dau2, relevant 
information such as copyright information is embedded as a watermark by the watermarking method indicated by the 

so used method information. 

[0171] Although the output digital data from the insertion unit 1 13 is not shown in figure 2, in this first embodiment 
the data structure of the output digital data is identical to that of the information-embedded digital data Dau. That is, the 
output digital data Is composed of a fixed-length first data section and a variable-length second data section, and the 
used method information indicating the used watermarking method is inserted in the digital data corresponding to the 

55 first data section while the relevant information (information to be embedded) is embedded in the digital data corre- 
sponding to the second data section by the used watermarking method. 

[0172] While in this first embodiment the first data section in the information-embedded digital data or the output 
digital data has a fixed length, the first data section may have a variable length when attribute information indicating the 
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attribute of the digital data is included in the digital data. In this case, however, size information indicating the size of the 
first data section must be inserted in the attribute information. 

[0173] Further, also when the information-embedded digital data or the output digital data includes information indi- 
cating the end point of the first data section, the first data section may have a variable length. 
5 [0174] Moreover, in the information-embedded digital data or the output digital data, the first and second data sec- 
tions are arranged such that the fixed-length first data section is followed by the variable -length second data section. 
However, these data sections may be arranged such that the variable-length second data section is followed by the 
fixed-length first data section. Moreover, a plurality of fixed-length first data sections and a plurality of variable-length 
second data sections may be dispersed in the digital data. 

10 

(data structure of used method information) 

[0175] Next, a description will be given of the data structure of the used method information lum indicating the used 
watermarking method. 

15 [0176] This used method information is embedded in the first data section of the information-embedded digital data 
Dau by a versatile watermarking method (e.g., the watermarking method A). The used method information is 2-bit data. 
In the control unit 1 1 5 of the data processing apparatus 1 1 0, according to the value of this data, the used method infor- 
mation indicating the watermarking method used for embedding the relevant information in the supply audio data is 
obtained. 

20 [0177] Figure 3 shows the values of the first and second bits of the two-bit data as the used method information. 
[0178] The value "00" of the used method information indicates that the input digital data has no embedded infor- 
mation according to the watermarking method. Further, the values °01 ", "10", and "1 1 " of the used method information 
indicate that the input digital data has embedded information according to the watermarking methods A, B, and C, 
respectively. 

25 [0179] Hereinafter, the operation of the data processing apparatus 1 10 will be described, 
(acquisition of digital data) 

[0180] Initially, in the data processing apparatus 1 1 0, the data obtaining unit 1 1 1 obtains desired digital data from 
30 the network signal Sn on the network N. It is assumed that the desired digital data is the information-embedded digital 
audio data (distributed audio data) Dau, in which relevant information such as copyright information has been embed- 
ded, as a watermark, in supply audio data to be supplied from the distribution end to the user, by a predetermined 
watermarking method (e.g., the watermarking method A). 

[0181] To be specific, the network signal (transmission signal) Sn corresponding to the desired digital data, which 
35 has been transmitted from the network N through the network cable, is received by the network interface 1 1 1a of the 
data obtaining unit 111. The received network signal Sn is converted to the desired digital data Dau by decoding in the 
received signal decoder 111b. The desired digital data Dau so obtained is transmitted through the data bus Dbus to the 
extraction unit 112, the insertion unit 113, and the control unit 115. 

[0182] While in this first embodiment all of the digital data obtained by the data processing apparatus 1 10 are sub- 
40 jected to the data conversion process, only some of the obtained digital audio data selected by the user of the apparatus 
may be subjected to the data conversion process. 

[0183] While in this first embodiment the digital data received by the data processing apparatus 110 is only the 
information-embedded digital data Dau, the data processing apparatus 11 0 may receive other kinds of digital data than 
the information-embedded digital data Dau as long as the apparatus 1 1 0 can identify the received digital data as digital 
45 data to be subjected to the data conversion process. 

(acquisition of used method information) 

[0184] Then, the information-embedded digital data Dau obtained by the data obtaining unit 1 11 is transmitted 
so through the data bus Dbus to the extraction unit 1 1 2 and the control unit 1 15. In the control unit 1 15, since the informa- 
tion-embedded digital data Dau is composed of the fixed-length first data section Dau1 and the variable-length second 
data section Dau2, the first data section is distinguished from the second data section, and the extraction unit 112 
extracts the used method information from the first data section in accordance with a control signal from the control unit 
115. 

55 [0185] To be specific, in the extraction unit 1 12, the used method information which has been embedded by the pre- 
determined watermarking method is extracted from the first data section Dau1 of the information-embedded digital data 
Dau, according to the control signal from the control unit 1 1 5. Since the two-bit data having the value of "01 ° is Included 
as the used method information in the digital data corresponding to the first data section, the extraction unit 1 12 obtains 
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the used method information indicating the used watermarking method A from the first data section. The used method 
information so obtained is stored in the execution memory of the control unit 115. 

[0186] While in this first embodiment the predetermined watermarking method is the versatile watermarking 
method A, any of the methods A, B, and C may be used for embedding the used method information, according to the 
5 construction of the control unit 115. Further, an arbitrary watermarking method other than these methods may be 
employed. 

[01 87] Further while in this first embodiment the first data section contains only the used method information, it may 
contain other information. 

io (extraction of embedded information) 

[0188] In the second data section Dau2 of the information -embedded digital data Dau transmitted from the data 
obtaining unit 1 1 1 through the data bus Dbus to the extraction unit 1 12, the relevant information has been embedded 
by the watermarking method A which is indicated by the obtained used method information. 
is [0189] In the extraction unit 1 12, the relevant information (embedded information) is extracted from the information- 
embedded digital data Dau in accordance with the used method information stored in the execution memory of the con- 
trol unit 1 15. The relevant information so extracted is once stored in the execution memory of the control unit 1 15. 

(re-embedding of embedded information) 



[0190] In the insertion unit 113, the relevant information which is stored in the execution memory of the control unit 
1 15 is embedded again in the information-embedded digital data Dau from the data obtaining unit 1 11 , by the prede- 
termined watermarking method, whereby output digital data is created. At this time, the relevant information which was 
embedded in the second data section Dau2 of the information -embedded digital data Dau is again embedded in the 
25 second data section Dau2, by the predetermined watermarking method. Further, during this information re-embedding 
process, the used method information indicating the used watermarking method is inserted in the first data section 
Dau1 of the information-embedded digital data Dau so as to define that the information is embedded by the predeter- 
mined watermarking method. In this case, the used watermarking method is the predetermined watermarking method. 
Then, the output digital data so created is supplied to the recording unit 1 14 through the data bus Dbus. 



(recording of output digital data) 

[0191] When the output digital data is supplied from the insertion unit 1 13 through the data bus Dbus to the record- 
ing unit 1 14, the output digital data is stored in the DVD- RAM disk in the recording unit 1 14. Since a file system is con- 
35 structed in the DVD-RAM disk, the output digital data is given an arbitrary file name and recorded in the DVD-RAM disk. 
[0192] Although the digital audio data is recorded with an arbitrary file name by utilizing the file system of the DVD- 
RAM disk, the recording manner is not restricted thereto. 

[0193] For example, the digital audio data may be simply recorded from the head of the recording area of the DVD- 
RAM medium. Further, the digital audio data may be written in a vacant area of the DVD-RAM disk which is detected at 
40 the time of recording the data in the disk. Furthermore, the digital audio data may be written in an area of the DVD-RAM 
disk which is specified by the user of the apparatus. 

[0194] While in this first embodiment the file name used when the digital audio data is recorded in the DVD-RAM 
disk is an arbitrary one, the user may specify the file name. Further, a file name described in attribute information which 
follows or is included in the digital data, may be used. 
45 [0195] As described above, the data processing apparatus 110 according to this first embodiment is provided with 
the data obtaining unit 1 1 1 which obtains the information-embedded digital data Dau in which the relevant information 
such as copyright information is embedded as a watermark, from the network signal Sn on the network N, and the con- 
trol unit 115 which obtains the used method information indicating the watermarking method which has been used 
when embedding the copyright information or the like. According to the obtained used method information, the embed- 
so ded information such as copyright information is extracted from the information-embedded digital data Dau by using an 
appropriate watermarking method, and the extracted information such as copyright information is embedded again in 
the information-embedded digital data Dau by using a predetermined watermarking method, thereby creating the out- 
put digital data to be recorded on the recording medium. Therefore, regardless of the watermarking method by which 
the relevant information has been embedded in the received digital data Dau, a reproduction apparatus as a domestic 
55 electrical apparatus can appropriately reproduce the information-embedded digital data Dau recorded on the recording 
medium, according to the embedded intonation such as copyright information. 

[0196] To be specific, since the used watermarking method information is embedded in the information-embedded 
digital data in which the relevant information has been embedded by a watermarking method, the data processing 
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apparatus which obtains this information-embedded digital data can distinguish the watermarking method used for 
embedding the relevant intonation in the digital data. Thereby, in the data processing apparatus, extraction or removal 
of the embedded intonation can be performed without extracting wrong information by using another method or without 
changing a portion other than the embedded information by using another method. 

5 [0197] As the result, at the distribution end from which digital data such as audio data is distributed to the user, the 
relevant information can be embedded as a watermark in the supply digital data to be supplied, by using a desired 
watermarking method. In other words, especially when the digital data is recorded on a recording medium at the user 
end, the distribution end of the digital data can select a watermarking method which provides the least degradation due 
to the information embedding process. Further, the distribution end can select the optimum method according to the 

io communication rate or the like when transmitting the digital data. 

[0198] Further, when the information-embedded digital data obtained from the network N is recorded on a writable 
recording medium, a watermarking method, which is different from the watermarking method used for embedding infor- 
mation in the supply digital data at the distribution end, can be used for embedding information in the digital data to be 
written in the recording medium. Therefore, information can be embedded in digital data by a watermarking method 

75 adapted to an apparatus which reproduces the digital data. 

Embodiment 2. 

[0199] Figure 4 is a block diagram for explaining a data processing apparatus 1 20 according to a second embodi- 

20 ment of the present invention. 

[0200] This data processing apparatus 120 obtains, from a network signal Sn on a network N, information-embed- 
ded digital audio data (distributed audio data) Dau as input audio data, and audio data attribute Information (hereinafter 
also referred to simply as attribute information) indicating the attribute of the distributed audio data Dau. On receipt of 
these data, the data processing apparatus 120 embeds information in the Input audio data and reproduces the input 

25 audio data, in accordance with control information included in the attribute information. 

[0201] The Information-embedded digital audio data Dau is obtained, at the distribution end, by embedding infor- 
mation in digital audio data to be supplied from the distribution end to the user (supply audio data) by using various 
watermarking methods. The audio data attribute information is transmitted from the distribution end together with the 
information-embedded digital audio data Dau. There are several transmission modes for the audio data attribute infor- 

30 mation, for example, the attribute information is arranged in a predetermined position (fixed position) in the audio data 
Dau to be transmitted, or the attribute information is multiplexed in the audio data Dau to be transmitted, or the attribute 
information is transmitted as a data file separated from the audio data Dau to be transmitted. 

[0202] Hereinafter, an electronic distribution system for audio data, including the data processing apparatus 120 of 
this second embodiment, will be described in detail. 
35 [0203] In the electronic distribution system, the information-embedded digital audio data Dau and the correspond- 
ing audio data attribute information are electronic-distributed from the distribution end by the network signal Sn through 
the network N. 

[0204] The audio data attribute information has a data structure including at least used watermarking method infor- 
mation (used method information). The used method information indicates the watermarking method which has been 
40 used for embedding relevant information in the supply audio data at the distribution end (used watermarking method). 
It is needless to say that the audio data attribute information may include other information in addition to the used 
method information. 

[0205] Further, when employing a specific method for constructing watermarking methods, even if plural pieces of 
information are embedded in one piece of target audio data by using the corresponding watermarking methods, the plu- 
45 ral pieces of information do no affect each other. For example, when information is embedded in a high-frequency 
domain of the target audio data by using a watermarking method while information is embedded in a low-frequency 
domain of the target audio data by using another watermarking method, the embedded two pieces of information do not 
affect each other. 

so (data structure of used watermarking method information) 

[0206] In this second embodiment, in order to cope with the above -described case, the used watermarking method 
information has a data structure which enables setting of method use status information indicating whether or not each 
of plural watermarking methods has been used for embedding information in the supply audio data. 
55 [0207] Figure 5 is a diagram illustrating the data structure of the used watermarking method information lum. 

[0208] This used watermarking method information lum is composed of plural pieces of method use status infor- 
mation lum(1), lum(2) f .... lum(n) corresponding to plural watermarking methods (n pieces of watermarking methods) 
which may be used in the information embedding process at the distribution end. 
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[0209] To be specific, method (1) use status information ium(1) indicates whether a watermarking method (1) has 
been used for embedding information in the supply audio data. The value "1" of this information lum(1) indicates that 
the watermarking method (1) has been used for the information embedding process at the distribution end, while the 
value "0" thereof indicates that the watermarking method (1 ) has not been used for the information embedding process 

5 at the distribution end. Likewise, method (2) use status information lum(2), .... method (n) use status information lum(n) 
indicate whether watermarking methods (2), (n) have been used for embedding information in the supply audio data. 

The value "1" of the information lum(2), lum(n) indicates that the corresponding watermarking methods (2) (n) 

have been used for the information embedding process at the distribution end while the value "0" of the information 
lum(2), lum(n) indicates that the corresponding watermarking methods (2) (n) have not been used for the infor- 

w mation embedding process at the distribution end. 

(data structure of embedded information) 

[0210] The embedded information in the information-embedded digital audio data includes at least reproduction 
75 permission information. 

[0211] This reproduction permission information indicates whether reproduction of the information-embedded dig- 
ital audio data Dau obtained as input digital audio data is permitted or not. The value "1* of this information indicates 
that reproduction of the audio data Dau is permitted while the value "0" of this information indicates that reproduction of 
the audio data Dau is not permitted. 

20 

(construction of data processing apparatus) 

[0212] The data processing apparatus 120 of this second embodiment includes a data obtaining unit 121 which 
obtains, from the network signal Sn on the network N, the information-embedded digital audio data (distributed audio 
25 data) Dau as input digital audio data, and the audio data attribute information which is transmitted together with the dis- 
tributed audio data Dau; and an extraction unit 122 which extracts the embedded information from the digital audio data 
Dau according to a control signal. 

[021 3] The data obtaining unit 121 comprises a network interface 121a which exchanges the network signal Sn sig- 
nal with the network N; and a received signal decoder 121b which decodes the network signal Sn supplied from the 

30 interface 121a to generate the information-embedded digital audio data Dau and the audio data attribute information.. 

Further, the extraction unit 122 comprises a method (1) extractor 121a1, a method (2) extractor 121a2 a method V*'/: <■ 

extractor 121 an corresponding to the above -described watermarking methods (1), (2), .... (n), respectively, and the 
embedded information is extracted from the information-embedded digital audio data Dau by a desired extractor in 
accordance with a control signal. 

35 [0214] For example, the method (1 ) extractor 1 21 a1 extracts the embedded information from the digital audio data 
Dau by using the watermarking method (1). The method (2) extractor 121 a2 extracts the embedded information from 
the digital audio data Dau by using the watermarking method (2). The method (n) extractor 121 an extracts the embed- 
ded information from the digital audio data Dau by the watermarking method (n). 

[0215] Further, the data processing apparatus 120 includes an insertion unit 1 24 which embeds the extracted infor- 
40 mation again in the information-embedded digital data Dau by using a predetermined watermarking method employed 
in reproduction apparatuses as domestic electrical equipment to generate output digital data; and a recording unit 125 
which stores the output digital data. 

[0216] Further, the data processing apparatus 120 includes a reproduction unit 123 which reproduces the informa- 
tion-embedded digital audio data Dau obtained in the data obtaining unit 121 , according to a control signal; and a con- 

45 trol unit 126 which controls the respective units 121 -125 by using the corresponding control signals. The reproduction 
unit 123 comprises an audio decoder 123a which converts the information-embedded digital data Dau to an audio sig- 
nal, and a speaker 123b which converts the audio signal to sound. Further, the control unit 126 comprises a CPU which 
performs various kinds of arithmetic processing and data processing, and a RAM (Random Access Memory) which 
stores the operation program of the CPU, thereby controlling the operations of the respective units 121-125. The RAM 

so also serves as an execution memory for temporarily storing the audio data or the audio data attribute information. 

[0217] To be specific, the control unit 126 decides the value of each method use status information in accordance 
with the used watermarking method information which is included in the audio data attribute information obtained by the 
data obtaining unit 121, and controls the process of extracting the embedded information by the method extractor cor- 
responding to each watermarking method, in accordance with the result of the decision. Further, the control unit 1 26 

55 controls the process of reproducing the audio data Dau by the reproduction unit 1 23, in accordance with the reproduc- 
tion permission information included in the extracted information. For example, when the values of the reproduction per- 
mission information included in all of the method use status information constituting the used watermarking method 
information are "1", the control unit 126 permits the reproduction unit 123 to reproduce the audio data Dau, and controls 
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the reproduction unit 123 so that it reproduces the audio data Dau. 

[0218] Further, in the data processing apparatus 120, the respective units are connected with each other by a data 
bus Dbus, and the control unit 126 controls not only the extraction unit 122 and the reproduction unit 123 but also the 
data obtaining unit 121 , the insertion unit 124, and the recording unit 125. 
5 [021 9] Further, the extraction unit 1 22 and the insertion unit 1 24 are implemented by custom LSIs, respectively. The 
recording unit 125 is implemented by a DVD-RAM drive, wherein the output digital audio data is recorded in a DVD- 
RAM disk. 

[0220] Hereinafter, a description will be given of the operation of the data processing apparatus 1 20. 

io (acquisition of audio data Dau and attribute information Dpr) 

[0221 ] When a network signal Sn supplied from the network N through the network cable is received by the network 
interface 1 21 a of the data obtaining unit 1 21 , the received network signal Sn is output to the received signal decoder 
121 b. The signal decoder 1 21 b decodes the network signal Sn to generate the information-embedded digital audio data 
15 Dau and the audio data attribute information Dpr which follows the audio data Dau. 

[0222] The audio data Dau so obtained is sent to the extraction unit 1 22 and the insertion unit 1 21 through the data 
bus Dbus, while the audio data attribute information Dpr is sent to the control unit 126 through the data bus Dbus. In the 
control unit 126, the audio data attribute information Dpr is temporarily stored in the RAM serving as an execution mem- 
ory. 

20 

(extraction of embedded information) 

[0223] The control unit 126 controls the extraction unit 122 by a control signal, in accordance with the used water- 
marking method information included in the audio data attribute information Dpr. That is, according to the value of each 
25 of the method use status information included in the used watermarking method information, the control unit 126 out- 
puts a control signal to the corresponding method extractor. 

[0224] To be specific, in the control unit 126, initially, the value of the method (1) use status information in the used 
watermarking method information is referred to. When the value of the method (1 ) use status information Is °1 °, the con- 
trol unit 126 outputs a control signal which instructs the method (1) extractor 122a1 in the extraction unit 122 to extract 

30 the embedded information from the input audio data Dau by using the watermarking method (1). Thereby, in the method 
(1) extractor 122a1, the information which has been embedded by the watermarking method (1) is extracted from the 
audio data Dau by the watermarking method (1). The information so extracted is temporarily stored in the execution 
memory of the control unit 126 via the data bus Dbus. On the other hand, when the value of the method (1) use status 
data is B 0\ the control unit 126 does not output a control signal which instructs extraction of the embedded information 

35 from the input audio data Dau, to the method (1 ) extractor 1 22a1 . 

[0225] Next, in the control unit 126, the value of the method (2) use status information included in the used water- 
marking method information is referred to. When the value of the method (2) use status data is "1% the control unit 126 
outputs a control signal which instructs the method (2) extractor 122a2 in the extraction unit 122 to extract the embed- 
ded information from the input audio data Dau by the watermarking method (2). Thereby, in the method (2) extractor 

40 122a2, the information which has been embedded by the watermarking method (2) is extracted from the input audio 
data Dau by the watermarking method (2). The information so extracted is temporarily stored in the execution memory 
of the control unit 126 via the data bus Dbus. On the other hand, when the value of the method (2) use status informa- 
tion is "0", the control unit 126 does not output a control signal which instructs extraction of the embedded information 
from the input audio data Dau, to the method (2) extractor 122a2. 

45 [0226] In this way, in the control unit 126, the process of supplying a control signal to the corresponding method 
extractor according to the value of the method use status data is performed for each of the plural pieces of method use 
status information (method (1) use status information ~ method (n) use status information) which may be used at the 
distribution end, and accordingly, each method extractor extracts the embedded information from the input audio data 
Dau in response to the control signal from the control unit 1 26. Thereby, plural pieces of embedded information are 

so extracted. 

(reproduction of audio data Dau) 

[0227] Thereafter, the control unit 1 26 decides the values of the reproduction permission information included in all 
55 of the information obtained by the above-described information extraction process. According to the result of the deci- 
sion, the control unit 126 outputs a control signal indicating whether the input digital audio data Dau is to be reproduced 
or not, to the reproduction unit 123. 

[0228] To be specific, when the values of the reproduction permission information included in all of the method use 
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status information constituting the used watermarking method information are "1", the control unit 126 permits the 
reproduction unit 123 to reproduce the audio data Dau. Then, under control of the control unit 126, the audio data Dau 
is transmitted from the data obtaining unit 121 through the data bus Dbus to the reproduction unit 123. In the reproduc- 
tion unit 123, the received audio data Dau is decoded by the audio decoder 123a to be converted to an audio signal, 
5 and sound corresponding to this audio signal is output from the speaker 123b. 

(re-embedding of embedded information) 

[0229] Further, in the data processing apparatus 120, independently of the process of reproducing the audio data 
io Dau obtained by the data obtaining unit 121 , the information (embedded information) extracted from the audio data Dau 
is again embedded in the audio data Dau by a predetermined watermarking method. The predetermined watermarking 
method is a watermarking method employed in reproduction apparatuses as domestic electrical equipment. 
[0230] As described above, in the control unit 126, the watermarking method which has been used for embedding 
information in the target audio data at the distribution end is decided according to the value of each method use status 
is data included in the used watermarking method information, and according to the result of the decision, the information 
embedded by the corresponding watermarking method is supplied to the insertion unit 124. In the insertion unit 124, 
the information supplied from the control unit 126 is again embedded in the audio data Dau supplied from the data 
obtaining unit 121, by the predetermined watermarking method, thereby generating output digital audio data. 

20 (recording of output digital audio data) 

[0231] The output digital audio data generated in the insertion unit 124 is supplied to the recording unit 125 and 
recorded in a recording medium, according to a control signal from the control unit 126. The recording unit 125 is imple- 
mented by a DVD-RAM drive, and the output digital audio data is stored in a DVD-RAM disk. 

25 [0232] In this second embodiment, the control unit 126 permits the reproduction unit 123 to reproduce the audio 
data Dau only when the values of the reproduction permission information included in all of the method use status infor- 
mation constituting the used watermarking method information are "1 B . However, the condition under which the control 
unit 126 permits the reproduction unit 123 to reproduce the audio data Dau is not restricted thereto. 
[0233] For example, reproduction of the audio data Dau may be permitted when the value of at least one piece of 

30 reproduction permission information included in the method use status information is "1". 

[0234] Alternatively, permission for reproduction of the audio data Dau may be decided as follows. That is, a weight- 
ing factor is set for each of the method use status information constituting the used watermarking method information, 
and the product of this weighting factor and the value of the reproduction permission information included in each 
method use status information is obtained, and then the products obtained for all of the method use status information 

35 are summed up. According to whether this sum exceeds a predetermined threshold or not, reproduction of the audio 
data Dau is permitted. 

[0235] As described above, according to the second embodiment of the present invention, the data processing 
apparatus 120 is provided with the data obtaining unit 121 which receives a network signal Sn from the network N and 
obtains the information-embedded digital audio data Dau and the audio data attribute information Dpr. The embedded 

40 information, which has been embedded in the audio data Dau by various watermarking methods, is extracted from the 
audio data Dau according to the method use status information included as the used watermarking method information 
in the audio data attribute information Dpr, and reproduction of the audio data Dau is controlled according to each 
extracted information. Therefore, in the data processing apparatus 120 receiving the audio data Dau, reproduction of 
the audio data Dau can be correctly controlled according to the embedded information which has been embedded in 

45 the audio data Dau by various watermarking methods at the distribution end. 

[0236] Furthermore, since the extracted information is again embedded in the audio data Dau by a predetermined 
watermarking method and then recorded in the recording medium, even if the information has been embedded by using 
various watermarking methods at the distribution end of the audio data Dau, the embedded information in the audio 
data Dau corresponding to the respective watermarking methods can be converted to information corresponding to the 

so predetermined watermarking method. Thereby, as in the first embodiment of the invention, even in a reproduction appa- 
ratus as a domestic equipment which employs the predetermined watermarking method, the information-embedded 
digital audio data supplied from the network can be reproduced satisfactorily according to the embedded information 
which has been embedded by the various watermarking methods. 

[0237] While in this second embodiment, the reproduction unit 1 23 reproduces the obtained information-embedded 
55 digital audio data (distributed audio data) Dau, the reproduction unit 123 may reproduce the data which is obtained by 
re-embedding the embedded information in the distributed audio data Dau. 

[0238] In this second embodiment, the information-embedded digital data may be obtained by embedding plural 
pieces of relevant information in the supply digital data by using a plurality of watermarking methods, and the insertion 
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unit may embed a predetermined number of relevant information amongst the plural pieces of relevant information, in 
the input digital data, by using the corresponding watermarking methods amongst the plural watermarking methods or 
using a watermarking method which is different from these watermarking methods, thereby generating the output digital 
data. 



Embodiment 3. 

[0239] Figure 6 is a block diagram for explaining a data processing apparatus 130 according to a third embodiment 
of the present invention. 

10 [0240] The data processing apparatus 1 30 obtains information -embedded audio data (distributed audio data) Dau 
which has been obtained by embedding information in digital audio data to be supplied from the distribution end to the 
user (supply audio data) by using various watermarking methods, from a network signal Sn on a network N or various 
kinds of data recording media. Then, the apparatus 130 decides whether the information has been embedded in the 
distributed audio data Dau by a predetermined watermarking method or not and, according to the result of this decision, 

15 embeds the information again in the distributed audio data Dau. 

[0241] The predetermined watermarking method is a versatile method employed for reproduction apparatuses 
which are domestic electrical equipment. Further, in the data processing apparatus 130, when the embedded informa- 
tion in the obtained audio data Dau has been embedded by the predetermined watermarking method, this audio data 
Dau is recorded in a data recording medium as it is. On the other hand, when the embedded information in the obtained 

20 audio data Dau has been embedded by a watermarking method other than the predetermined watermarking method, 
the embedded information is extracted from the audio data Dau by using the corresponding watermarking method. 
Then, the extracted information is again embedded in the audio data Dau by using the predetermined watermarking 
method, and the resultant audio data is recorded in the recording medium. 

[0242] Hereinafter, a description will be given of an audio data supply system including the data processing appa- 
25 ratus 130 of this third embodiment. 

[0243] In the supply system, there are two cases where the audio data has is supplied by electronic distribution 
from the network N and where it is supplied by distribution from the data recording medium. 

(data structure of audio data) 



[0244] In this third embodiment, the audio data to be supplied has a data structure including no information relating 
to a watermarking method. Therefore, the data processing apparatus 130 distinguishes the watermarking method 
which has been used for embedding information in the supplied audio data, according to the route through which the 
audio data is obtained. 

35 [0245] To be specific, for digital data like the above-mentioned audio data, a watermarking method is decided In 
advance according to its supply mode, and so the watermarking method which has been used for embedding informa- 
tion in this data can be specified by specifying the data obtaining route. 

[0246] For example, when the audio data is supplied through a network, a watermarking method according to a pro- 
tocol used for communication is employed. Further, when the audio data is supplied from a recording medium, a water- 

40 marking method according to the type of the recording medium is employed. Therefore, the audio data supply system 
of this third embodiment does not require the following methods: the method of inserting the information indicating the 
watermarking method corresponding to the information-embedded digital audio data to be electronic-distributed (used 
watermarking method information) according to the first embodiment; and the method of inserting the used watermark- 
ing method information in the audio data attribute information which follows the digital audio data to be electronic-dis- 

45 tributed according to the second embodiment. 

(construction of data processing apparatus) 

[0247] The data processing apparatus 130 of this third embodiment includes a data obtaining unit 131 and an 
so extraction unit 132. The data obtaining unit 131 obtains, as the distributed audio data, the information-embedded digital 
audio data from the network signal Sn on the network N, and the information-embedded digital audio data from the data 
recording medium. The extraction unit 132 extracts the embedded information from the distributed audio data in accord- 
ance with a control signal. 

[0248] The data obtaining unit 131 comprises a network Interface 131a which exchanges the network signal Sn 
55 with the network N; a received signal decoder 131b which decodes the network signal Sn from the interface 131a to 
generate the information-embedded digital audio data (distributed audio data); a DVD-ROM drive 131c which reads the 
Information-embedded digital audio data from a DVD-ROM disk; and a semiconductor memory access unit 1 31 d which 
reads the information-embedded digital audio data from a semiconductor recording medium. 
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[0249] Further, the extraction unit 132 comprises a method (A) extractor 1 32a, a method (B) extractor 132b, and a 
method (C) extractor 132c which correspond to the watermarking methods (A), (B), and (C), respectively. In the extrac- 
tion unit 132, the embedded information in the information-embedded digital audio data is extracted by a desired extrac- 
tor according to a control signal. 
5 [0250] For example, the method (A) extractor 1 32a extracts the embedded information from the informati on-embed- 
ded digital audio data by using the watermarking method (A). The method (B) extractor 132b extracts the embedded 
information from the information-embedded digital audio data by using the watermarking method (B). The method (C) 
extractor 1 32c extracts the embedded information from the information -em bedded digital audio data by using the water- 
marking method (C). 

10 [0251] Further, the data processing apparatus 130 includes an insertion unit 133 which embeds the information 
extracted by the extraction unit 132 again in the information-embedded digital audio data by using a predetermined 
watermarking method employed in reproduction apparatuses as domestic electrical equipment to generate output 
audio data; and a recording unit 134 which stores the output audio data. The insertion unit 133 includes a method (A) 
insertion unit 1 33a which embeds the extracted information in the information -embedded digital audio data by using the 

is watermarking method (A). 

[0252] Further, the data processing apparatus 130 includes a control unit 135 which controls the respective units 
131-134 by the corresponding control signals. In this data processing apparatus 130, the respective units 131-135 are 
connected with each other through a data bus Dbus. 

[0253] The extraction unit 132 and the insertion unit 133 are implemented by custom LSI, respectively. The record- 
er? ing unit 134 is provided with a DVD- RAM drive, wherein the output digital audio data is recorded in a DVD- RAM disk. 
[0254] The control unit 135 comprises a CPU which performs various kinds of arithmetic processing and data 
processing, and a RAM (Random Access Memory) which contains the operation program for the CPU. The control unit 
135 controls the operations of the respective units 131 -134. The RAM also serves as an execution memory which tem- 
porarily stores the audio data output from the respective units, and the used watermarking method information. 
25 [0255] Further, the control unit 135 has a table 135a in which the data obtaining routes are correlated with the 
watermarking methods. This table 135a is stored in a nonvolatile memory in the control unit 135. The control unit 135 
further includes a method intonation decision means which decides the intonation indicating the watermarking method 
which has been used when embedding the information in the obtained audio data (used watermarking method informa- 
tion) in accordance with the intonation indicating the route through which the audio data has been obtained (route infor- 
30 mation); and a re-insertion decision means which decides whether the extracted intonation is to be inserted again in the 
obtained audio data or not, in accordance with the used watermarking method information. 

[0256] The watermarking method used for the digital audio data obtained from the network signal Sn on the net- 
work N depends on intonation relating to a server connected to the network N through which the audio data is obtained 
(URL: Uniform Resource Locator), a communication protocol used when the audio data is obtained, and the like. Fur- 
as ther, the watermarking method for the digital audio data stored in the data recording medium depends on the type of 
the data recording medium. So, if the data obtaining route is found, the control unit 135 can decide one watermarking 
method corresponding to the audio data by referring to the table 135a. In this third embodiment, the data obtaining unit 
131 decides the audio data obtaining route, and information indicating the audio data obtaining route is supplied from 
the data obtaining unit 131 to the control unit 135. 
40 [0257] Furthermore, in this third embodiment, the watermarking method B corresponds to the audio data obtained 
from the network signal Sn on the network N, the watermarking method A corresponds to the audio data obtained from 
the DVD-ROM, and the watermarking method C corresponds to the audio data obtained from the semiconductor 
recording medium. Further, the predetermined watermarking method which is employed in reproduction apparatuses 
or the like as domestic electrical equipment, is the watermarking method A. 
45 [0258] While in this third embodiment a DVD-ROM disk and a semiconductor memory are employed as data 
recording media, other media such as a DVD-RAM disk and a CD-ROM disk may be employed. Since a predetermined 
watermarking method is used for each of these recording media, the receiver of digital data stored in each of these 
recording media can distinguish the corresponding watermarking method according to the data obtaining route. 
[0259] Next, the operation of the data processing apparatus 130 will be described. 

50 

(acquisition of digital audio data) 

[0260] In the data processing apparatus 130 of this third embodiment, digital audio data Is obtained from the net- 
work signal Sn on the network N or from the recording medium. For example, the network signal Sn on the network N 
55 is transmitted through the network cable and received by the network interface 1 31 a of the data obtaining unit 1 31 . The 
received network signal Sn is decoded by the received signal decoder 131 b, thereby generating the information-embed- 
ded digital audio data (distributed audio data) Dau. On the other hand, the digital audio data recorded in the DVD-ROM 
disk is read by the DVD-ROM drive 131c in the data obtaining unit 131. Further, the digital audio data recorded in the 
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semiconductor memory is read by the semiconductor memory access unit 131d in the data obtaining unit 131. 
[0261] The distributed audio data obtained in the data obtaining unit 131 is supplied to the extraction unit 132 and 
the insertion unit 133. Further, this digital audio data is supplied to the control unit 135, together with the route informa- 
tion indicating the route through which it is obtained. In the control unit 135, the digital audio data is stored in the exe- 
5 cution memory. 

(acquisition of used watermarking method information) 

[0262] In the control unit 135, it is decided which of the plural watermarking methods on the table 135a has been 
io used for embedding the information in the digital audio data supplied from the data obtaining unit 1 35, based on the 
route information, by referring to the correspondence of the data obtaining routes and the watermarking methods on the 
table 135a. The result of this decision is temporarily stored in the execution memory of the control unit 135. 

(extraction of embedded information) 

15 

[0263] In the extraction unit 132, the embedded information is extracted from the digital audio data obtained in the 
data obtaining unit 131 , by a watermarking method according to a control signal from the control unit 135. 
[0264] For example, when it is decided that the obtained digital audio data corresponds to the watermarking 
method A, the extraction unit 132 is controlled so that extraction of the embedded information from the digital audio data 
20 is performed by the watermarking method A. To be specific, extraction of the embedded information is performed by the 
method A extractor 132 under control of the control signal from the control unit 135. 

[0265] Further, when it is decided in the control unit 135 that the obtained digital audio data corresponds to the 
watermarking method B, in the extraction unit 1 32, extraction of the embedded information is performed by the method 
B extractor 132b. When it is decided that the obtained digital audio data corresponds to the watermarking method C, 
25 extraction of the embedded information is performed by the method C extractor 132c. 

(decision about re-embedding of embedded information) 

[0266] When the digital audio data obtained in the data obtaining unit 131 corresponds to the watermarking method 
30 A, the control unit 135 decides that it is not necessary to embed the information embedded in this digital audio data 
again in this digital audio data by using another watermarking method. The reason is as follows. The information 
embedded in this digital audio data can be extracted by a reproduction apparatus employing the predetermined water- 
marking method (in this case, method A), for example, an existing reproduction apparatus for playing a DVD-RAM disk. 
[0267] On the other hand, when the digital audio data obtained in the data obtaining unit 131 corresponds to the 
35 watermarking method B, the control unit 135 decides that it is necessary to embed the embedded information again in 
the digital audio data by the watermarking method A. 

(re-embedding of embedded information) 

40 [0268] When the digital audio data corresponds to the watermarking method A and the control unit 135 decides that 
re-embedding of information in the digital audio data is not necessary, no control signal is supplied from the control unit 
135 to the insertion unit 133, and the digital audio data stored in the execution memory of the control unit 135 is sup- 
plied to the recording unit 134. 

[0269] On the other hand, when the digital audio data corresponds to the watermarking method B and the control 
45 unit 135 decides that re-embedding of information in the digital audio data is necessary, in the insertion unit 133, the 
information embedded in the information-embedded digital audio data (distributed audio data) is again embedded in the 
distributed audio data by the watermarking method A in accordance with a control signal from the control unit 135. 

(recording of digital audio data) 

50 

[0270] The digital audio data which is decided in the control unit 1 35 as corresponding to the watermarking method 
A is recorded in the recording unit 134 without being subjected to the reembedding process in the insertion unit 133. 
On the other hand, the digital audio data which is decided in the control unit 135 as corresponding to a watermarking 
method other than the method A is subjected to the re-embedding process in the insertion unit 133 and, thereafter, 

55 — stored in the recording unit 134. - - 

[0271 ] As described above, according to the third embodiment of the invention, the data processing apparatus 1 30 
includes, in addition to the constituents of the apparatus 1 10 of the first embodiment, the control unit 135 having the 
means for receiving the digital audio data in which information is embedded by a watermarking method, and deciding 
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which of plural watermarking methods has been used for this digital audio data, according to the information indicating 
the route through which the audio data is obtained. Therefore, it is not necessary to provide the audio data to be trans- 
mitted from the distribution end to the network N with the corresponding watermarking method information. Further, it 
is not necessary to provide the audio data to be stored in the recording medium with the corresponding watermarking 
5 method information. 

[0272] Further, when the distributed audio data corresponds to a predetermined watermarking method employed 
in reproduction apparatuses or the like as domestic electrical equipment, this audio data is recorded as it is in the 
recording medium. On the other hand, when the distributed audio data corresponds to a watermarking method other 
than the predetermined method, the information embedded in the distributed audio data is again embedded in this 
io audio data by the predetermined watermarking method, and the thus obtained data is recorded in the recording 
medium. Therefore, the digital audio data for which the re-embedding process is not required is prevented from being 
subjected to the information writing process again, thereby minimizing the processing for the digital audio data which 
causes degradation of sound quality. 

is Embodiment 4. 

[0273] Figure 7 is a block diagram for explaining a data processing apparatus 140 according to a fourth embodi- 
ment of the present invention. 

[0274] The data processing apparatus 1 40 reads multiplexed digital data obtained by multiplexing digital video data 
20 and digital audio data, from a data recording medium, and extracts embedded information from the digital audio data 
by using a watermarking method. Then, the apparatus 1 40 displays the extracted information as an image, and embeds 
the extracted information again in the digital audio data. In this fourth embodiment, the digital audio data included in the 
multiplexed digital data is data obtained by embedding relevant information in supply audio data to be supplied from the 
distribution end to the user by using a watermarking method (distributed audio data). Further, the extraction of the 
25 embedded information is performed by using an appropriate watermarking method, and the re-embedding process is 
performed by using a predetermined watermarking method which is employed in reproduction apparatuses or the like 
as domestic electrical equipment. 

[0275] The appropriate watermarking method is identical to the watermarking method which has been used for 
embedding the relevant information in the supply audio data. Further, the multiplexed digital data corresponds to one 
30 program or one movie, and this is called a video object. 

[0276] In the following description of the fourth embodiment, the digital video data and the digital audio data are 
referred to as video data and audio data, respectively. 

(data structure of video object) 

35 

[0277] Figure 8 shows the data structure of the video object. 

[0278] In this fourth embodiment, the video object corresponds to one program stream in MPEG-2 (Moving Picture 
Experts Group Phase 2), and it is composed of plural video object units (VOBU). 

[0279] Further, in a portion of the multiplexed digital data corresponding to one VOBU, video data and audio data 
40 are multiplexed by time division multiplexing. The digital data corresponding to one VOBU has a data quantity equiva- 
lent to a reproduction time required for about 15 frames (= 0.5 sec). In this fourth embodiment, the unit of time division 
for data in one VOBU is 2048 bytes, and a portion comprising 2048 bytes of data is called a packet. Further, a packet 
comprising video data is called a video packet while a packet comprising audio data is called an audio packet. 
[0280] For example, the video object Vob is composed of the first VOBU(#1 ) Vob1 ~ the n-th VOBU(#n) Vobn, and 
45 the first VOBU(#1) Vob1 is composed of the first video packet (#1)Pv1 ~ the k-th video packet (#k)Pvk, and the first 
audio packet (#1 )Pa1 ~ the m-th audio packet (#m)Pam. 
[0281] Figure 9 shows the data structure of an audio packet. 

[0282] An audio packet Pa is composed of a packet header Pah and an audio data section Pad. An audio stream, 
which is a code string constituting audio data, is stored in the audio data section Pad, and information is embedded in 
so the audio stream by an appropriate watermarking method. On the other hand, various kinds of information required for 
reproducing the audio data are stored in the packet header Pah. 

[0283] To be specific, the packet header Pah contains a packet type flag Fpk, reproduction time information Irt, sub- 
stream ID information Iss, and used watermarking method information I urn. 

[0284] The packet type flag Fpk is information for identifying the type of the corresponding packet. The reproduction 
55 time information Irt indicates the time required for reproducing the data stored in the corresponding packet, and a rela- 
tive time based on the reproduction time for the data at the head of the video object is used as the reproduction time. 
[0285] Further, in the video object as multiplexed digital data, since plural kinds of audio streams can be multi- 
plexed, information for identifying each audio stream is required. So, the packet header Pah contains the substream ID 
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information Iss for identifying each of the plural kinds of audio streams included in the video object. For example, the 
substream ID information Iss indicates a serial number set for each audio stream. 

[0286] Furthermore, the used watermarking method information lum indicates a watermarking method by which 
information has been embedded in the audio streams stored in the audio data section Pad of the audio packet Pa. 
5 [0287] Figure 1 0 shows the values stored in the packet header Pah as the used watermarking method information 
lum. 

[0288] In figure 1 0, the used watermarking method information lum is represented by 2-bit values (1 st bit value and 
2nd bit value). The value °00" of this method information lum indicates that the watermarking method A has been 
employed for embedding information in the audio data. Likewise, the values "01 "1 0", and "1 1" of the method informa- 
io tion lum indicate that the watermarking methods B, C, and D have been employed for embedding information in the 
audio data. 

[0289] In the data structure of the video object according to this fourth embodiment, the used watermarking method 
information corresponding to the supply audio data is stored in the audio packets constituting the video object. However, 
the used watermarking method information may be stored in the video packets or other kinds of packets. 
is [0290] Further, although the used watermarking method information corresponding to the supply audio data is 
stored in the packet header Pah of each audio packet Pa, the storage location of the used watermarking method infor- 
mation is not restricted thereto. 

[0291] For example, the used watermarking method information may be stored in some audio packets amongst the 
audio packets constituting the video object. 
20 [0292] Further, the used watermarking method information may be stored in a recording area of a DVD-ROM disk, 
separately from the video object. Further, the used watermarking method information may be embedded in the audio 
streams stored in the audio data section, by an appropriate method. 

[0293] In this fourth embodiment, the audio packet Pa contains, in its packet header Pah, the packet type flag Fpk, 
the reproduction time information Irt, the substream ID information Iss, and the used watermarking method information 
25 lum. However, other kinds of information than the above-mentioned Fpk, Irt, Iss and lum may be stored in the packet 
header pah of the audio packet Pa. 



(construction of data processing apparatus 140) 

30 [0294] The data processing apparatus 140 includes a data obtaining unit 141 which has a DVD-ROM for reading 
the video object from a DVD-ROM disk, and obtains the video object according to a control signal; and a separation unit 
142 which is constituted by a custom LSI, and separates video data and audio data from the obtained video object, 
according to a control signal. 

[0295] Further, the data processing apparatus 140 includes an extraction unit 143 which is constituted by a custom 
35 LSI, and extracts embedded information from the used watermarking method information lum that is stored in the 

header Pah of each audio packet Pa of the separated audio data, according to a control signal; and an insertion unit 

146 which is constituted by a custom LSI, and embeds the extracted information again in the separated audio data by 

using a predetermined watermarking method, according to a control signal, thereby generating output audio data. 

[0296] The extraction unit 143 comprises a method A extractor 143a, a method B extractor 143b, a method C 
40 extractor 143c, and a method D extractor 143d which correspond to the above-mentioned watermarking methods (A), 

(B), (C), and (D), respectively. In the extractor unit 143, the embedded information in the input audio data is extracted 

by a desired extractor, according to a control signal. 

[0297] For example, the method A extractor 1 43a extracts the embedded information from the input audio data (dis- 
tributed audio data) by the watermarking method A. The method B extractor 143b extracts the embedded information 
45 from the input audio data by the watermarking method B. The method C extractor 1 43c extracts the embedded infor- 
mation from the input audio data by the watermarking method C. The method D extractor 1 43d extracts the embedded 
information from the input audio data by the watermarking method D. 

[0298] The data processing apparatus 140 further includes a video reproduction unit 144 which performs image 
display of the separated video data, and image display of the information such as copyright information extracted from 

so the audio data, according to a control signal; and an audio reproduction unit 145 which reproduces the output audio 
data. The video reproduction unit 144 comprises a video decoder 144a which converts the video data to an analog 
video signal by signal processing such as decoding, and decodes the information such as copyright information sup- 
plied from the extraction unit 143; and a television receiver 1 44b which performs image display of the input analog video 
signal, and image display of the decoded information. Further, the audio reproduction unit 1 45 comprises an audio 

55 decoder 145a which decodes the audio data to output an audio signal, and a speaker 145b which converts the audio 
signal to sound to be output 

[0299] The data processing apparatus 140 further includes a recording unit 147 which records the output audio 
data from the insertion unit 146 according to a control signal; and a control unit 1 48 which comprises a CPU performing 
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various kinds of arithmetic processing and data processing and a RAM (Random Access Memory) for storing the oper- 
ation program of the CPU. The control unit 148 controls the operations of the respective units 141-147 by using the 
control signals. The RAM of this control unit 148 also serves as an execution memory for temporarily storing the data 
from the separation unit 142, the information from the extraction unit 143, and the data from the insertion unit 146. 

5 [0300] While in this fourth embodiment the audio reproduction unit 1 45 reproduces the obtained input audio data 
(distributed audio data), the audio reproduction unit 145 may reproduce the output audio data which is obtained by 
embedding the embedded information again in the input audio data by using a predetermined watermarking method. 
[0301] Further, in this fourth embodiment, the data processing apparatus 140 includes the data obtaining unit 141 
having a DVD-ROM drive. However, the data obtaining unit 141 may have any storage unit, such as a DVD-RAM drive, 

10 a CD-ROM drive, or a semiconductor memory reading unit, as long as it is a bulk storage unit capable of handling mul- 
timedia data. 

[0302] Further, in this fourth embodiment, the data processing apparatus 140 includes, as units for reproducing the 
obtained video object, both of the video reproduction unit 144 for reproducing video data and the audio reproduction 
unit 1 45 for reproducing audio data. However, the data processing apparatus 1 40 may includes one of the video repro- 
75 duction unit 144 and the audio reproduction unit 145. Further, the apparatus 140 may include, besides these units 144 
and 145, a unit for reproducing data such as subtitle. 

[0303] Further, in this fourth embodiment, the separation unit 142, the extraction unit 143, and the insertion unit 144 
are constituted by custom LSI circuits, respectively. However, these units 142-144 may be constituted by a CPU per- 
forming various kinds of arithmetic processing and a RAM for storing the operation program of the CPU and also serv- 
20 ing as an execution memory. In this case, these units 142-144 can share the constituents with the control unit 148. 
[0304] Hereinafter, the operation of the data processing apparatus 140 will be described. 

[0305] In the following description, for simplification, it is assumed that the video object is continuously recorded in 
the DVD-ROM drive, starting from the head of the data recording area of the disk. 

25 (reading of video object) 

[0306] In the data obtaining unit 1 41 , the multiplexed digital data as the video object are successively read in packet 
units, from the head of the reading area of the DVD-ROM disk, in accordance with the control signal from the control 
unit 148, and the read data are successively transmitted to the separation unit 142 in packet units. 

30 

(separation of video object) 

[0307] In the separation unit 1 42, the packet type flags Fpk which are stored in the packet headers of the respective 
packets (video packets and audio packets) are read. Then, the video packets and the audio packets are identified 
35 according to the packet type flags Fpk, and these packets are separated from the video object. Thereafter, under control 
of the control unit 148, the packets decided as being video packets are sent to the video reproduction unit 144 and the 
control unit 148, while the packets decided as being audio packets are sent to the extraction unit 143, the insertion unit 
146, the audio reproduction unit 145, and the control unit 148. 

40 (extraction of embedded information) 

[0308] In the control unit 148, the used watermarking method information Irm which is stored in the packet header 
Pah of the audio packet Pa (refer to figure 9) is read. Then, the value of the used watermarking method information lum 
is decided, and a control signal according to the value of the method information is output to the extraction unit 143. In 
45 the extraction unit 1 43, according to the control signal, embedded information is extracted from the audio stream stored 
in the audio data section Pad of the audio packet Pa, by a method extractor according to an appropriate watermarking 
method. The embedded information includes information corresponding to a copyright display character string of the 
copyright bolder. 

[0309] For example, when the value of the used watermarking method information lum is "00", the method A extrac- 
50 tor 143a extracts the embedded information by using the watermarking method A, according to the control signal. Like- 
wise, when the value of the used watermarking method information lum is °01", "10", or "11", the method B extractor 
143b, the method C extractor 143c, or the method D extractor 143d extracts the embedded information by using the 
watermarking method B, C, or D, according to the control signal, respectively. 

55 (reproduction of video data) 

[0310] When the video packets separated In the separation unit 142 are supplied to the video reproduction unit 1 44, 
the video decoder 144a performs MPEG2 system decoding on the video data stored in each video packet, which data 
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has been subjected to MPEG2 system coding, according to the control signal from the control unit 148. When a video 
signal obtained by the decoding process in the video decoder 144a is input to the TV receiver 144b, the video signal is 
reproduced and displayed. 

5 (display of copyright information) 

[031 1] When the copyright display character string information (embedded information) is supplied from the extrac- 
tion unit 143 to the video reproduction unit 144, this information is converted to a display signal by the decoder 144a to 
be output to the TV receiver 144b. In the TV receiver 144b, the copyright display character string information is dis- 
10 played so that it is laid over the display image corresponding to the video signal. 

[0312] Although the copyright display character string information as copyright information is laid over the display 
image of the video signal, it may be displayed by display means other than the TV receiver. Image display of the copy- 
right information is not necessarily performed. 

is (reproduction of audio data) 

[0313] When the audio packet Pa separated from the video object by the separation unit 142 is supplied to the 
audio reproduction unit 145, the audio decoder 145a performs LPCM decoding on the audio streams stored in the audio 
data section Pad of the audio packet Pa (i.e., audio data which has been subjected to LPCM coding), thereby generat- 
20 ing an audio signal. This audio signal is converted to sound by the speaker 145b to be output. 

[0314] In the data processing apparatus 140, while the above-described reproduction of the video data and the 
audio data is performed, re-embedding of embedded information in the distributed audio data and recording of the out- 
put audio data obtained by the re-embedding process are carried out in like manner as described for the first to third 
embodiments. 



(decision about re-embedding of information) 

[031 5] When the audio data in the audio packet Pa supplied from the separation unit 1 42 corresponds to the water- 
marking method A, the control unit 148 decides that it is not necessary to embed the information embedded in this dig- 
30 ital audio data again in this digital audio data by using another watermarking method. The reason is as follows. The 
information embedded in this audio data can be extracted by a reproduction apparatus which employs the predeter- 
mined watermarking method (in this case, method A), for example, an existing reproduction apparatus for a DVD- RAM 
disk. 

[0316] On the other hand, when the audio data in the audio packet Pa supplied from the separation unit 142 corre- 
al sponds to the watermarking method B, the control unit 1 48 decides that it is necessary to embed the embedded infor- 
mation again in the digital audio data by using the watermarking method A. 

(re-embedding of embedded information) 

40 [0317] When the audio data corresponds to the watermarking method A and so the control unit 148 decides that 
re-embedding of information in the digital audio data is not necessary, no control signal is supplied from the control unit 
148 to the insertion unit 146, and the audio data stored in the execution memory of the control unit 148 is supplied to 
the recording unit 1 47. 

[0318] On the other hand, when the audio data corresponds to the watermarking method B and so the control unit 
45 148 decides that re-embedding of information in the digital audio data is necessary, in the insertion unit 146, the embed- 
ded information is again embedded in the audio data by the watermarking method A, according to the control signal 
from the control unit 148. 

(recording of digital audio data) 



[0319] The audio data which is decided in the control unit 148 as corresponding to the watermarking method A is 
recorded in the recording unit 147 without being subjected to the re-embedding process in the insertion unit 146. On 
the other hand, the audio data which is decided in the control unit 1 48 as corresponding to a watermarking method 
other than the method A is subjected to the re-embedding process in the insertion unit 146 and, thereafter, stored in the 
55 recording unit 147. ... 

[0320] As described above, the data processing apparatus 140 of this fourth embodiment includes the data obtain- 
ing unit 141 having the DVD-ROM drive, instead of the data obtaining unit 111 of the apparatus 110 according to the 
first embodiment, and further, the apparatus 140 includes the separation unit 142 for separating video packets and 
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audio packets from the video object read from the DVD-ROM disk, and the video reproduction unit 144 for reproducing 
video data. In this construction, the embedded information extracted from the data section of each audio packet is dis- 
played in the video reproduction unit 144 in accordance with the used watermarking method intonation stored in the 
header of the audio packet, whereby the copyright information can be confirmed as an image. 

5 [0321] Further, when the digital audio data corresponds to a predetermined watermarking method which is 
employed in reproduction apparatuses or the like as domestic electrical equipment, this digital audio data is recorded 
as it is in the recording medium. On the other hand, when the digital audio data corresponds to a watermarking method 
different from the predetermined method, the information embedded in this audio data is embedded again in this audio 
data by using the predetermined watermarking method, and thus obtained data is recorded in the recording medium. 

io Therefore, the digital audio data for which the re-embedding process is not required is prevented from being subjected 
to the information writing process again, thereby minimizing the processing for the audio data, which causes degrada- 
tion of sound quality. 

[0322] In this fourth embodiment, the video object is recorded from the head of the recording area of the DVD-ROM 
disk. However, when auxiliary information such as management information is recorded at the head of the recording 

75 area of the DVD-ROM disk, the video object may be recorded after the auxiliary information. 

[0323] Further, while in this fourth embodiment the video object is continuously recorded in the recording area of 
the DVD-ROM disk, the recording form of the video object in the DVD-ROM disk is not restricted thereto. 
[0324] For example, when the management information includes TOC (Table of Contents) information which indi- 
cates the recording start position of the video object in the DVD-ROM disk and the recording size, the video object can 

20 be dispersed ly recorded in arbitrary positions in the recording area of the DVD-ROM disk. 

[0325] Further, in this fourth embodiment, the video object is sequentially read, starting from the head of the record- 
ing area of the DVD-ROM disk. However, when the management information includes scenario information indicating 
the access order to the respective recording areas in the disk, the video objects which are dispersedly stored in the plu- 
ral recording areas in the disk can be read by accessing the respective recording areas in an arbitrary order. Especially 

25 when the data processing apparatus 1 40 is provided with a function of accepting a signal under control of the user, the 
access order to the respective recording areas can be changed according to this signal when reading the video objects. 
[0326] Further, while in this fourth embodiment the audio packets separated from the video object in the separation 
unit are supplied directly to the audio reproduction unit, these audio packets may be transmitted through the extraction 
unit to the audio reproduction unit. 

30 [0327] While in this fourth embodiment only the audio data contains the embedded information, the video data may 
contain the embedded information. In this case, the data processing apparatus 140 is provided with a video extraction 
unit for extracting the embedded information from the video data, and the video packets separated from the video object 
are supplied to the video reproduction unit through this video extraction unit 

[0328] Further, while in this fourth embodiment the embedded information in the audio data includes the informa- 
35 tion Indicating the copyright display character string for the copyright holder, the embedded Information may include, 
besides the copyright information, information indicating the names of persons authorized to reproduce the audio data 
by the copyright holder, the dates of authorizations, etc. Especially when video object can be copied, the embedded 
information in the audio data may include copy generation information or the like. 

40 Embodiment 5. 

[0329] Figure 1 1 is a block diagram for explaining a data processing apparatus 1 50 according to a fifth embodiment 
of the present invention. 

[0330] The data processing apparatus 150 performs the following processes: an extraction process for obtaining, 
45 from a data recording medium, digital audio data (distributed audio data) which has been obtained by embedding infor- 
mation in digital audio data to be supplied from the distribution end to the user (supply) by using a watermarking 
method, together with information indicating the watermarking method used for the information embedding process, 
and then extracting the embedded information from the input audio data; a removal process for removing the embedded 
information from the input audio data to obtain processed audio data; a reproduction process for reproducing the proc- 
50 essed audio data; and a re-embedding process for embedding the extracted information again in the processed audio 
information by using a predetermined watermarking method. 

[0331] The data processing apparatus 150 is adaptable to plural watermarking methods because there is a case 
where plural watermarking methods are used for one audio data. 

[0332] To be specific, when using a specific method for constructing watermarking methods, even when plural 
55 pieces of information are embedded in one audio data by using the corresponding watermarking methods, the plural 
pieces of embedded information do not affect each other. For example, when the information is embedded in high-fre- 
quency components of the audio data by using a watermarking method while the Information Is embedded in low-fre- 
quency components of the audio data by using another watermarking method, the information embedded in the high- 
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frequency components and the information embedded in the low-frequency components do not affect each other. The 
data processing apparatus 150 of this fifth embodiment is adaptable to such a case. 

[0333] In this fifth embodiment, the watermarking method used for embedding information in the supply audio data 
is one of the four watermarking methods A, B, C, and D. Further, extraction of the embedded information is performed 
5 by using an appropriate watermarking method, and the re-embedding process is performed by using a predetermined 
watermarking method which is employed in reproduction apparatuses or the like as domestic electrical equipment. Fur- 
ther, the appropriate watermarking method described above is identical to the watermarking method used for embed- 
ding information in the supplied audio data. 

10 (data structure of distributed audio data) 

[0334] Figure 1 2 shows the data structure of the distributed audio data recorded in the data recording medium. 
[0335] The distributed audio data Dde comprises TOC (Table of Contents) information Ito indicating the contents of 
music, and various kinds of discrete audio data Da1~Dan which have been coded by the LPCM (Linear Pulse Code 
15 Modulation). 

[0336] The TOC information Ito is recorded at the head of the recording area of the data recording medium, and 
includes plural pieces of information to be used when reproducing the audio data Da1-Dan recorded in the recording 
medium. To be specific, the plural pieces of information used for reproducing the audio data are as follows: audio data 
recording start position information Isp, audio data recording end position information lep, music title information Ina, 
20 number-of-quantization information Iqu, sampling rate information Isa, number-of-channel information Ich, and used 
watermarking method information lum. 

[0337] In the recording area of the data recording medium, the TOC information Ito is followed by the audio data 
Da1 -Dan which have been coded by LPCM. 

[0338] While in this fifth embodiment LPCM is employed as the audio data coding method, other coding methods, 
25 such as AC3, MPEG -audio, and DTS (Digital Theater System) may be employed. The AC3 is an audio compressive 
coding method for processing movie audio (5.1 ch, 2ch, ect), and this method is used in DVD-VIDEO and the like. Like- 
wise, the DTS is an audio compressive coding method for processing movie audio (5.1 ch, 2ch, etc.), and this method 
is used in movie theaters and DVD-VIDEO. 

[0339] Figure 13 shows the data structure of the used watermarking method information lum. 

30 [0340] In this fifth embodiment, since the watermarking method used for embedding informtion in the supply audio 
data is any of the above-mentioned four methods (method A, method B, method C, and method D), the used water- 
marking method information lum is composed of four pieces of use status information lus corresponding to the respec- 
tive watermarking methods, i.e., method A use status information lusa, method B use status information lusb, method 
C use status information lusc, and method D use status information lusd. 

35 [0341] For each use status information, "0" or "1 ■ is set. 

[0342] For example, when the value of use status information is set at "1 this means that a watermarking method 
corresponding to this use status information has been used for embedding information in the supply audio data. On the 
other hand, when the value of use status information is set at "0", this means that a watermarking method correspond- 
ing to this use status information has not been used for embedding information in the supply audio data. 

40 [0343] The data recording medium may contain not only the TOC information and the audio data but also other 
information. Further, the TOC information may include not only the audio data reproduction information but also other 
information, such as information indicating the reproduction order of the audio data. Further, the audio data reproduc- 
tion information may include composer information, copyright information and the like, in addition to the audio data 
recording start position information Isp, the audio data recording end position information lep, the music title information 

45 Ina, the number-of-quantization information Iqu, the sampling rate information Isa, the number-of-channel information 
Ich, and the used watermarking method information lum. 

[0344] While in this fifth embodiment the audio data which has been coded is recorded on the data recording 
medium, the audio data recorded on the recording medium may be multiplexed with other data by a multiplexing method 
based on MPEG2 or the like. 

so [0345] Further, while in this fifth embodiment the TOC information is recorded at the head of the recording area of 
the data recording medium, the TOC information may be recorded in an arbitrary position in the data recording medium 
as long as the data reading means (in this case, data obtaining unit) can recognize the storage position of the TOC 
information in the data recording medium. 

55 (construction of data processing apparatus) 

[0346] The data processing apparatus 1 50 includes a data obtaining unit 151 which has a DVD-ROM drive for read- 
ing information or data recorded in a DVD-ROM disk, and obtains the TOC information Ito and the plural pieces of audio 
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data Da1-Dan from the DVD-ROM disk in accordance with a control signal; and a storage unit 152 which has a TOC 
information storage region 152a wherein the TOC information Ito obtained by the data obtaining unit 151 is stored, and 
stores the TOC information Ito in this storage region 152a in accordance with a control signal. The storage unit 152 is 
provided with a DRAM (Dynamic Random Access Memory). 

5 [0347] Further, the data processing apparatus 1 50 includes an extraction unit 1 53 which is constituted by a custom 
LSI and extracts embedded information from the audio data, with reference to the used watermarking method informa- 
tion lum included in the obtained TOC information Ito, according to a control signal; a removal unit 154 which is consti- 
tuted by a custom LSI and removes embedded information from the distributed audio data, with reference to the used 
watermarking method information lum included in the obtained TOC information Ito, according to a control signal, 

io thereby generating processed audio data; and an insertion unit 155 which is constituted by a custom LSI and embeds 
the extracted information again in the processed audio data by using a predetermined watermarking method, according 
to a control signal, thereby generating output audio data. 

[0348] The extraction unit 153 comprises a method A extractor 153a, a method B extractor 153b, a method C 
extractor 1 53c, and a method D extractor 153d which correspond to the above-mentioned watermarking methods A, B, 
75 C, and D, respectively. In the extractor unit 1 53, the embedded information in the distributed audio data is extracted by 
a desired extractor, according to a control signal. 

[0349] For example, the method A extractor 153a extracts the embedded information from the distributed audio 
data (input audio data) by the watermarking method A. The method B extractor 1 53b extracts the embedded informa- 
tion from the input audio data by the watermarking method B. The method C extractor 153c extracts the embedded 
20 information from the input audio data by the watermarking method C. The method D extractor 1 53d extracts the embed- 
ded information from the input audio data by the watermarking method D. 

[0350] Further, the removal unit 154 comprises a method A remover 154a, a method B remover 154b f a method C 
remover 154c, and a method D remover 154d which corresponds to the above-described watermarking methods A, B, 
C, and D, respectively. In the removal unit 154, the embedded information in the distributed audio data is removed by a 

25 desired remover according to a control signal. 

[0351] For example, the method A remover 154a removes the embedded information from the distributed audio 
data (input audio data) by the watermarking method A. The method B remover 154b removes the embedded informa- 
tion from the input audio data by the watermarking method B. The method C remover 154c removes the embedded 
information from the input audio data by the watermarking method C. The method D remover 154d removes the embed- 

30 ded information from the input audio data by the watermarking method D. 

[0352] The data processing apparatus 1 50 further includes an audio reproduction unit 1 56 for reproducing the proc- 
essed audio data according to a control signal, and a recording unit 157 for recording the output audio data according 
to a control signal. The audio reproduction unit 156 comprises an audio decoder 156a for decoding the output audio 
data to output an audio signal, and a speaker 156b for converting the audio signal to sound to be output In the audio 

35 reproduction unit 156, the discrete audio data are reproduced in order of the audio data reproduction intonation 
Da1 - Dan included in the TOC information. 

[0353] The data processing apparatus 150 further includes a control unit 158 which comprises a CPU performing 
various kinds of arithmetic processing and data processing and a RAM (Random Access Memory) for storing the oper- 
ation program of the CPU. The control unit 158 controls the operations of the respective units 151-157 by using the 
40 control signals. The RAM of this control unit 158 also serves as an execution memory for temporarily storing the audio 
data from the data obtaining unit 151 and the removal unit 154, and the extracted information from the extraction unit 
153. 

[0354] While in this fifth embodiment the audio reproduction unit 156 reproduces the processed audio data, the 
audio reproduction unit 1 56 may reproduce the output audio data which has been obtained by embedding the informa- 
45 tion embedded in the distributed audio data, again in the processed audio data, by using the predetermined watermark- 
ing method. 

[0355] Further, in this fifth embodiment, the data processing apparatus 150 includes the data obtaining unit 151 
having a DVD-ROM drive. However, the data obtaining unit 151 may have any storage unit, such as a DVD-RAM drive, 
a CD-ROM drive, or a semiconductor memory reading unit, as long as it is a bulk storage capable of handling multime- 



[0356] Further, while in this fifth embodiment the extraction unit 153, the removal unit 154, and the insertion unit 
155 are constituted by custom LSI circuits, respectively, each of these units 153-155 may be constituted by a CPU per- 
forming various kinds of arithmetic processing and a RAM for storing the operation program of the CPU and also serv- 
ing as an execution memory. In this case, these units 153-155 can share the constituents with the control unit 158. 
55 [0357] Further, while in this fifth embodiment the storage unit 152 is provided with a DRAM, it may be provided with 
any memory as long as it is suitable for temporary storage, for example, a SRAM may be employed. 
[0358] Hereinafter, the operation of the data processing apparatus 1 50 will be described. 

[0359] In the following description, for simplification, it is assumed that the TOC information Ito is continuously 
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recorded at the head of the recording area of the DVD-ROM disk, as information of a predetermined size. 



(reading of TOC information) 



5 [0360] In the data obtaining unit 151, information of a predetermined size is read from the head of the recording 
area of the DVD-ROM disk, according to a control signal from the control unit 158, and the read information, i.e., the 
TOC information Ito, is supplied to the storage unit 152. 

(storage of TOC information) 

10 

[0361] In the storage unit 152, according to a control signal from the control unit 158, the whole TOC information 
Ito obtained in the data obtaining unit 151 (i.e., all pieces of the audio data reproduction information IM~!rn included in 
the TOC information Ito) is stored as it is (i.e., without being processed) in the DRAM. Alternatively, a portion of the 
audio data reproduction information Ir1~lrn included in the TOC information Ito (e.g., only the audio data reproduction 
15 information corresponding to the audio data to be reproduced next) may be stored in the DRAM. 

(reading of audio data) 

[0362] Next, in the control unit 158, the audio data reproduction information Ir1~lrn corresponding to the discrete 
20 audio data Da1 -Dan to be reproduced are sequentially obtained from the TOC information Ito stored in the storage unit 
152, in reproduction order of the discrete audio data. 

[0363] Further, in the control unit 158, the audio data recording start position information Isp and the audio data 
recording end position information lep are obtained from the obtained audio data reproduction information. Then, a con- 
trol signal according to the audio data recording start position information Isp is supplied from the control unit 1 58 to the 
25 data obtaining unit 151 . In the data obtaining unit 151 , according to the control signal from the control unit 158, discrete 
audio data to be reproduced is obtained from the DVD-ROM disk. The discrete audio data so obtained is supplied to 
the extraction unit 1 53 and the removal unit 1 54. 

[0364] While the data obtaining process is being preformed, the control unit 158 monitors where the data is being 
read from the recording area of the DVD-ROM disk, and when the position where the data is being read agrees with the 
30 position indicated by the audio data recording end position information lep, the data obtaining unit 151 is controlled by 
the control unit 158 so as to stop reading the discrete audio data. ;.?V 
[0365] Thereafter, in the control unit 158, to obtain discrete audio data to be reproduced next, the audio data repro-' 
duction information corresponding to this audio data is obtained from the storage unit 1 52, followed by the process of 
obtaining the next discrete audio data. 

35 

(extraction of embedded information) 

[0366] In the control unit 158, the used watermarking method information is extracted from the audio data repro- 
duction information corresponding to the discrete audio data to be reproduced. Then, the value of the used status infor- 
40 mation corresponding to each watermarking method in the used watermarking method information lum is decided in 
the control unit 158. Then, a control signal according to the decided value of the use status information is output from 
the control unit 158 to the extraction unit 153. In the extraction unit 153, according to the control signal, the embedded 
information is extracted from the audio data to be reproduced, by an extractor corresponding to an appropriate water- 
marking method. 

45 [0367] For example, when the value of the method A use status information luma in the used watermarking method 
information lum is "1", the method A extractor 153a extracts the embedded information by using the watermarking 
method A, according to the control signal. On the other hand, when the value of the method A use status information 
luma is "0", extraction of the embedded information by using the method A extractor 153a is not carried out. Likewise, 
also the method B extractor 1 53b, the method C extractor 1 53c, and the method D extractor 154d perform extraction of 

so embedded information by using the corresponding watermarking methods B, C and D, according to the values of the 
method B use status information lumb, the method C use status information lumc, and the method D use status infor- 
mation lumd, respectively. 

(removal of embedded information) 

55 

[0368] The control unit 158 outputs a control signal according to the used watermarking method information 
obtained from the audio data reproduction information corresponding to the audio data to be reproduced, to the removal 
unit 154. In the removal unit 154, according to the control signal, embedded information is extracted from the audio data 
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to be reproduced by a method remover corresponding to an appropriate watermarking method. 
[0369] For example, in the control unit 158, the value of the method A use status information luma in the used 
watermarking method information lum is decided. Based on the result of the decision, when the value of the method A 
use status information luma is °*r, in the removal unit 154, the embedded information is removed by the watermarking 
5 method A f according to the control signal from the control unit 1 58. On the other hand, when the value of the method A 
use status information luma is "0*, removal of the embedded information by the watermarking method A is not per- 
formed in the removal unit 154. 

[0370] Likewise, in the control unit 158, the values of other method status information are successively decided. 
When the value is "1 the embedded information is removed by the method remover corresponding to the method use 
w status information whose value has been decided. On the other band, when the value is "0°, removal of the embedded 
information by the method remover corresponding to the method use status information whose value has been decided, 
is not performed. 

[0371] The distributed audio data which has been subjected to the removal process in the removal unit 154, i.e., the 
processed audio data, are supplied to the audio reproduction unit 156. 

75 

(reproduction of discrete audio data) 

[0372] The transmitted discrete audio data supplied to the audio reproduction unit 1 56 is subjected to LPCM decod- 
ing to be converted to an audio signal by the audio decoder 156a, and this audio signal is converted to sound by the 
20 speaker 156b. 

[0373] in the data processing apparatus 150, while reproducing the audio data as described above, re-embedding 
of information in the discrete audio data and recording of the output audio data obtained by the re-embedding process 
are carried out in like manner as described for the first to fourth embodiments. 

25 (decision about re-embedding of information) 

[0374] When the discrete audio data from the data obtaining unit 151 corresponds to the watermarking method A, 
the control unit 158 decides that it is not necessary to embed the information embedded in this discrete audio data 
again in this audio data by using another watermarking method. The reason is as follows. The information embedded 
30 in this discrete digital audio data can be extracted by a reproduction apparatus which employs the predetermined water- 
marking method (in this case, method A), for example, an existing reproduction apparatus for a DVD-RAM disk. 
[0375] On the other hand, when the discrete audio data from the data obtaining unit 151 corresponds to the water- 
marking method B f the control unit 158 decides that it is necessary to embed the embedded information again in this 
audio data by the watermarking method A. 

35 

(re-embedding of information) 

[0376] When the discrete audio data corresponds to the watermarking method A and so the control unit 158 
decides that re-embedding of information in this audio data is not necessary, no control signal is supplied from the con- 
40 trol unit 158 to the insertion unit 155, and the discrete audio data stored in the execution memory of the control unit 158 
is supplied to the recording unit 157. 

[0377] On the other hand, when the discrete audio data corresponds to the watermarking method B and so the con- 
trol unit 158 decides that re-embedding of information in this audio data is necessary, in the insertion unit 155, the 
embedded intonation is again embedded in the discrete audio data by the watermarking method A, according to the 
45 control signal from the control unit 1 58. 

(recording of discrete audio data) 

[0378] The discrete audio data which is decided by the control unit 1 58 as corresponding to the watermarking 
so method A is recorded in the recording unit 157 without being subjected to the re-embedding process in the insertion 
unit 155. On the other hand, the discrete audio data which is decided by the control unit 158 as corresponding to a 
watermarking method other than the method A is subjected to the re-embedding process in the insertion unit 155 and, 
thereafter, stored In the recording unit 157. 

[0379] As described above, the data processing apparatus 150 of this fifth embodiment includes, Instead of the 
55 data obtaining unit 1 1 1 of the apparatus 110 according to the first embodiment, the data obtaining unit 151 which has 
the DVD-ROM drive for reading the distributed audio data from the DVD-ROM disk, and obtains the TOC information Ito 
and plural pieces of discrete audio data Da1~Dan which are included in the audio data read from the DVD-ROM disk. 
According to the use watermarking method intonation lum which is stored in the TOC information Ito as the audio data 
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reproduction information Ir1 ~lrn corresponding to the respective discrete audio data, information embedded in the dis- 
crete audio data, such as copyright information, is removed from the audio data before reproducing the audio data. 
Therefore, reproduction of the discrete audio data distributed by the recording medium can be satisfactorily performed 
without degradation of sound quality due to the embedded information such as copyright information. 

5 [0380] Further, when the discrete audio data corresponds to a predetermined watermarking method which is 
employed in reproduction apparatuses or the like as domestic electrical equipment, this discrete audio data is recorded 
as it is in the recording medium. On the other hand, when the discrete audio data corresponds to a watermarking 
method different from the predetermined method, the information embedded in this audio data is again embedded in 
this audio data by using the predetermined watermarking method and thus obtained audio data is recorded in the 

w recording medium. Therefore, the discrete audio data for which the re-embedding process is not required is prevented 
from being subjected to the information writing process again, thereby minimizing the processing for the digital audio 
data, which causes degradation of sound quality. 

[0381] In this fifth embodiment, the TOC information is recorded at the head of the recording area of the DVD-ROM 
disk. However, if the control unit 158 can recognize the position where the TOC information is recorded in the recording 

15 area of the disk, the TOC information may be recorded in any position in the recording area. 

[0382] For example, when a file system is constructed on the DVD-ROM disk and information recorded on the disk 
is accessible according to its file name, the TOC information is given a file name when being recorded on the disk, 
whereby the TOC information can be read regardless of the physical location of the TOC information in the disk. 
[0383] Further, in this fifth embodiment, reproduction of the discrete audio data is carried out in the order of the 

20 audio data reproduction information corresponding to the respective discrete audio data recorded in the TOC informa- 
tion. However, when the data processing apparatus 150 has a function of accepting user's selection for discrete audio 
data to be reproduced next, the user can specify the discrete audio data to be reproduced next. 
[0384] Further, when information indicating the reproduction order of the discrete audio data (reproduction order 
information) is included in the TOC information, the control unit 158 can decide the reproduction order of the discrete 

25 audio data by referring to the reproduction order information. 

[0385] Further, in this fifth embodiment, the audio reproduction unit 156 decodes the supplied discrete audio data 
by the audio decoder 156a and reproduces the decoded data by the speaker 156b. However, the construction of the 
audio reproduction unit 156 is not restricted to this. For example, the audio reproduction unit having the audio decoder 
and the speaker may be constructed such that an analog audio signal obtained by decoding the discrete audio data by 

30 the audio decoder can be output to another recording unit through an audio cable or the like. 

[0386] Further, the data processing apparatus 150 of this fifth embodiment may be constructed such that the dis- 
crete audio data supplied to the audio reproduction unit is output, as being digital data, to another digital data recording 
unit through a digital transmission cable or the like. 

[0387] Further, in this fifth embodiment, the recording media (DVD-ROM disk) mounted on the DVD-ROM drive of 
35 the data obtaining unit 151 may be a recording medium in which the following data and information are recorded: infor- 
mation-embedded digital data obtained by embedding relevant information in supply digital data to be supplied from the 
distribution end to the user, and used method information indicating the watermarking method which has been used for 
embedding the relevant information in the supply digital data, and following the information-embedded digital data. In 
this case, the information-embedded digital data has the data structure which enables extraction or removal of the rel- 
40 evant information by using an appropriate watermarking method based on the used method information. Further, in the 
recording medium (DVD-ROM disk), plural pieces of used method information indicating different watermarking meth- 
ods which have been used for embedding the relevant information in the supply digital data, may be recorded. 

Embodiment 6. 

45 

[0388] Figure 14 is a block diagram for explaining a data processing apparatus 160 according to a sixth embodi- 
ment of the present invention. 

[0389] This data processing apparatus 160 receives a broadcast wave from a broadcast station and obtains, from 
the received broadcast wave, first digital audio data (distributed audio data) which has been obtained by embedding 

so information in digital audio data to be supplied from the distribution end to the user (supply audio data) by using a water- 
marking method. Then, the apparatus 160 performs extraction and removal of the embedded information (hereinafter 
also referred to as first embedded information) from the distributed audio data, and processes the extracted information 
(first embedded information) to create processed information (second information). Then, the apparatus 160 embeds 
the processed information in second digital audio data (processed audio data) obtained by the removal process per- 

55 formed on the distributed audio data, by using a watermarking method different from the above-mentioned watermark- 
ing method, thereby generating third digital audio data (output audio data). 

[0390] The watermarking method used for embedding information in the supply audio data is one of method A, 
method B, and method C. Further, the extraction of the first embedded information is performed by using an appropriate 
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watermarking method, and the re-embedding process is performed by using at least a predetermined watermarking 
method employed in reproduction apparatuses as domestic equipment. Further, the appropriate watermarking method 
is identical to the watermarking method used for embedding information in the supplied audio data. The distributed 
audio data has been subjected to LPCM coding, and the first embedded information has been embedded in the distrib- 
5 uted audio data by a predetermined watermarking method (method A). 

(data structure of first embedded information) 

[0391] Figure 1 5a shows the data structure of the first embedded information Iwm1 . 

10 [0392] The first embedded information Iwm1 contains copyright holder information Icr, information of a watermark- 
ing method used at recording Irm (hereinafter referred to simply as recording watermarking method information Irm), 
recording apparatus ID information Idi, copy generation information leg, and copy source ID information Ics. 
[0393] The first embedded information Iwm1 has a fixed data length. So, in the distributed audio data whose data 
length varies according to the length of music or the like, the same information is repeatedly embedded as the first 

15 embedded information. 

[0394] Figure 16(a) shows the values of the recording watermarking method information Irm included in the first 
embedded information Iwm1. 

[0395] The recording watermarking method information Irm is information for identifying the watermarking method 
which is used for embedding information in audio data to be recorded in the data processing apparatus 160. 

20 [0396] To be specific, the first embedded information Iwm1 has been embedded in the distributed audio data by a 
predetermined watermarking method. When the distributed audio data or processed audio data obtained by processing 
the distributed audio data is recorded on the recording medium, information corresponding to the first embedded infor- 
mation is again embedded by another watermarking method. The recording watermarking method information Irm is 
information for identifying the watermarking method to be used for this re-embedding process. 

25 [0397] The recording watermarking method information Irm is represented by two bits. More specifically, the value 
"00" of this Information Irm indicates that no information embedding process by a watermarking method is to be per- 
formed on the audio data to be recorded in the recording medium. The value "01" of this information Irm indicates that 
information is to be embedded in the audio data to be recorded, by using the watermarking method A. Likewise, the 
values "10" and "1 1" of this information Irm indicate that information is to be embedded in the audio data to be recorded, 

30 by using the watermarking method B and C, respectively. 

[0398] While in this sixth embodiment the LPCM is employed as the audio data coding method, other coding meth- 
ods, such as AC3, MPEG-audio, and DTS, may be employed. 

[0399] Further, while in this sixth embodiment audio data is independently transmitted by a broadcast wave to the 
data processing apparatus 1 60, the transmission mode of audio data is not restricted to this. For example, a video 
35 object (multiplexed digital data) in which video data and audio data are multiplexed may be transmitted to the apparatus 
160. 

[0400] Further, while in this sixth embodiment the distributed audio data is generated by embedding information in 
the supply audio data by a predetermined watermarking method, if information indicating the kind of the watermarking 
method used for the information embedding process is transmitted to the data processing apparatus 160, any water- 
40 marking method can be employed in the information embedding process. 

[0401] Further, while in this sixth embodiment the same information is repeatedly embedded as the first embedded 
information Iwm1 in the distributed audio data, different kinds of information may be embedded as the first embedded 
information Iwm1. 

[0402] Moreover, while in this sixth embodiment the first embedded information Iwm1 includes the copyright holder 
45 information Icr, the recording watermarking method information Irm, the recording apparatus ID information Idi, the copy 
generation information leg, and the copy source ID information Isc, other information, such as recording date informa- 
tion Ird, may be included in the first embedded information Iwm1. 

(data structure of second embedded information) 

50 

[0403] Figure 15(b) shows the data structure of the second embedded information Iwm2 which is embedded in the 
audio data to be recorded on the recording medium (distributed audio data or processed audio data) by a watermarking 
method. 

[0404] The second embedded information Iwm2 includes used watermarking method information I urn, instead of 
55 the recording watermarking method information Irm included in the first information Iwm1. The used watermarking 
method information lum indicates the watermarking method which has been used for embedding information in the dis- 
tributed audio data or the processed audio data. The values of the used watermarking method information lum and the 
meaning of each value are identical to those already described for the recording watermarking method information Irm. 
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[0405] That is, as shown In figure 1 6(b), the used watermarking method information ium is represented by two bits. 
The value "00" of this information Ium indicates that no information embedding process by a watermarking method has 
been performed on the audio data to be recorded in the recording medium. The value "01" of this information Ium indi- 
cates that information has been embedded in the audio data to be recorded, by using the watermarking method A. Like- 
5 wise, the values "1 0" and "1 1 ■ of this information Ium indicate that information has been embedded in the audio data to 
be recorded, by using the watermarking method B and C, respectively. 

[0406] While in this sixth embodiment the data structure of the second embedded information lwm2 is identical to 
that of the first embedded information Iwm1 , the second embedded information Iwm2 may have a data structure in 
which information of another title, such as copy date information, is added to the first embedded information, or a data 
10 structure in which part of the first embedded information, such as the copy generation information, is deleted from the 
first embedded information. 

[0407] Further, while in this sixth embodiment the used watermarking method information Ium is included in the 
second embedded information Iwm2, when attribute information of audio data is recorded in the recording medium sep- 
arately from the distributed audio data or the processed audio data, the used watermarking method information Ium 
is may be recorded as the attribute information. In this case, no used watermarking method information is included in the 
second embedded information Iwm2. 

(construction of data processing apparatus) 

20 [0408] The data processing apparatus 160 includes a data obtaining unit 161 comprising an antenna 161a which 
receives a broadcast wave Bw from a broadcast station Be, and a receiving decoder 161b which decodes the received 
broadcast wave Bw to generate the above-described distributed audio data (first digital audio data). The data obtaining 
unit 161 obtains the distributed audio data from the broadcast wave Bw according to a control signal. 
[0409] The data processing apparatus 1 60 further includes an extraction unit 1 62 which is constituted by a custom 

25 LSI and extracts the embedded information (first embedded information Iwm1 ) from the distributed audio data, accord- 
ing to a control signal; a removal unit 163 which is constituted by a custom LSI and removes the embedded information 
Iwm1 from the distributed audio data according to a control signal, thereby generating processed audio data (second 
digital audio data); and an insertion unit 1 64 which is constituted by a custom LSI and embeds processed information 
(second embedded information Iwm2) obtained by processing the first embedded information Iwm1, again in the proc- 

30 essed audio data, by using various watermarking methods, thereby generating output audio data (third digital audio 
data). 

[0410] The extraction unit 1 62 comprises a method A extractor 162a, a method B extractor 1 62b, and a method C 
extractor 162c which correspond to the above-mentioned watermarking methods A, B, and C, respectively. In the 
extractor unit 162, the embedded information Iwm1 in the distributed audio data is extracted by a desired extractor, 

35 according to a control signal. 

[0411] For example, the method A extractor 162a extracts the embedded information Iwm1 from the distributed 
audio data (input audio data) by the watermarking method A. The method B extractor 1 62b extracts the embedded 
information Iwm1 from the input audio data by the watermarking method B. The method C extractor 162c extracts the 
embedded information lwm1 from the input audio data by the watermarking method C. 

40 [0412] Further, the removal unit 163 comprises a method A remover 163a, a method B remover 163b, and a 
method C remover 1 63c which corresponds to the above-described watermarking methods A, B, and C, respectively. 
In the removal unit 163, the embedded information Iwm1 in the distributed audio data is removed by a desired remover 
according to a control signal. 

[0413] For example, the method A remover 163a removes the embedded information Iwm1 from the distributed 
45 audio data by the watermarking method A. The method B remover 1 63b removes the embedded information Iwm1 from 
the distributed audio data by the watermarking method B. The method C remover 163c removes the embedded infor- 
mation Iwm1 from the distributed audio data by the watermarking method C. 

[0414] The data processing apparatus 150 further includes a recording unit 1 66 which stores the output audio data 
according to a control signal; an apparatus information storage unit 1 65 which stores apparatus number information for 

so identifying this apparatus 160; and a control unit 167 which comprises a CPU performing various kinds of arithmetic 
processing and data processing and a RAM (Random Access Memory) for storing the operation program of the CPU. 
The control unit 167 controls the operations of the respective units 161-166 by using the control signals. 
[0415] The apparatus information storage unit 165 is provided with an EEPROM, and the apparatus number infor- 
mation is stored in an apparatus number storage region 165a of the EEPROM. Further, the recording unit 166 is pro- 

55 vided with a DVD-RAM drive, wherein the output audio data is recorded on a DVD-RAM disk. Further, the control unit 
1 67 processes the first embedded information in accordance with the apparatus number information stored in the 
recording area of the EEPROM, thereby producing the second embedded information. Further, the RAM of the control 
unit 1 67 also serves as an execution memory for temporarily storing the audio data supplied from the removal unit 163 
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and the insertion unit 164, the first embedded information Iwm1 extracted by the extraction unit 162, and the second 
embedded information Iwm2 generated in the control unit 167. 

[0416] While in this sixth embodiment the data obtaining unit 161 includes the antenna and the decoder, the con- 
struction of the data obtaining unit 161 is not restricted thereto. 
s [0417] For example, the data obtaining unit 1 61 may have a receiving unit which receives a transmitted signal via 
a cable line, such as a network interface or a modem. Further, the data obtaining unit 1 61 may have a bulk storage unit 
capable of accessing multimedia data, such as a DVD-ROM drive, a DVD-RAM drive, a CD-ROM drive, or a semicon- 
ductor memory access unit. 

[0418] Further, in this sixth embodiment, the extraction unit 162, the removal unit 163, and the insertion unit 164 
10 are constituted by separated custom LSI circuits. However, since these units perform similar processes, these units are 
desired to be constituted by a single custom LSI circuit, for reduction in the number of components. 
[0419] Further, while in this sixth embodiment the extraction unit 162, the removal unit 163, and the insertion unit 
164 are constituted by custom LSI circuits, each of these units 162-1 64 may be constituted by a CPU performing vari- 
ous kinds of arithmetic processing and a RAM for storing the operation program of the CPU and also serving as an exe- 
15 cution memory. In this case, these units 1 62-1 64 can share the constituents with the control unit 1 67. 

[0420] Further, while in this sixth embodiment the apparatus information storage unit 1 65 is equipped with an EEP- 
ROM, the storage unit 166 may be equipped with any memory as long as it is a nonvolatile memory. 
[0421] Further, while the apparatus information storage unit 1 65 stores the apparatus number information for iden- 
tifying the apparatus, it may store an identification number for identifying the user of the apparatus. In this case, the 
20 apparatus information storage unit is equipped with a reader for an IC card or a magnetic card, and the user of the 
apparatus inserts the card in which the user's identification number is recorded, into the reader to inform the apparatus 
of the user's identification number. 

[0422] While in this sixth embodiment the recording unit 166 is equipped with a DVD-RAM drive, the recording unit 
166 may be equipped with any storage unit, such as a CD-R drive or a semiconductor memory access unit, as long as 
25 it is a bulk writable storage capable of accessing multimedia data. 

[0423] Hereinafter, a description will be given of the operation of the data processing apparatus 1 60. 

(reading of audio data) 

30 [0424] J- ip .tbe. data^obtaining. jur>it 1^61 .Aty* broadcast wave Bw from the broadcast station Be is received by the 
antenna i o¥'d;. "'tFf tyo^C i -t|t «ve-&$ *;j input to the decoder 161b, it is converted to the distributed audio data 
(first digital audftf akrkf by decoding. l his distributed audio data is supplied to the extraction unit 162 and the removal 
unit 163. 

35 (extraction of embedded Information) 

[0425] In the extraction unit 162, the first embedded information Iwm1 is extracted from the distributed audio data 
obtained in the data obtaining unit 161, by using a predetermined watermarking method, according to a control signal 
from the control unit 167. The extracted information (first information Iwm1) is temporarily stored in the execution mem- 
40 ory of the control unit 1 67. 

(removal of embedded information) 

[0426] In the removal unit 163, the first embedded information Iwm1 is removed from the distributed audio data, by 
45 using a predetermined watermarking method, according to a control signal from the control unit 1 67, thereby generating 
processed audio data. The processed audio data is supplied to the insertion unit 162. 

(processing of embedded information) 

so [0427] In the control unit 1 67, the first information lwm1 stored in the execution memory is processed to create the 
second information Iwm2. 

[0428] To be specific, the second information Iwm2 is created according to the copyright holder information, the 
recording watermarking method information, the recording apparatus ID information, the copy generation information, 
and the copy source ID information which are included in the first information Iwm1 stored in the execution memory, and 
55 the apparatus number information stored in the apparatus information storage 1 65. 

[0429] At this time, with respect to the copyright holder information and the recording watermarking method infor- 
mation, the values described in the first information Iwm1 are copied as they are into the second information Iwm2. As 
the recording apparatus ID information of the second information Iwm2, the apparatus number recorded in the appara- 
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tus information storage unit 1 65 is recorded as it is. Thereby, the possessor of the copied audio data can be specified. 
Further, in the second information Iwm2, as the value of the copy generation information, a value obtained by adding 1 
to the value recorded in the first information Iwm1 , is recorded. Thereby, when a copy of the distributed audio data is 
found, the generation of this copy, i.e., how many times the original audio data has been copied, can be specified. 
5 [0430] Furthermore, as the copy source ID information of the second information Iwm2, the recording apparatus ID 
information in the first information Iwm1 is recorded. Thereby, the apparatus which has copied the distributed audio data 
can be specified. By tracing back the copy source ID information, the user or apparatus who/which has illegally copied 
the audio data can be specified. 

[0431] The second information Iwm2 so created is sent to the insertion unit 1 64. 

10 

(re-embedding of second information) 

[0432] In the insertion unit 164, the second information Iwm2 generated in the control unit 167 is embedded in the 
processed audio data supplied from the removal unit 1 63, by using a watermarking method, according to a control sig- 

15 nal, thereby generating output audio data (third digital audio data). 

[0433] At this time, the second information Iwm2 is repeatedly embedded over the processed audio data, from the 
head to the end, according to the length of the audio data. Especially, the first embedding of the second information 
Iwm2 is performed by the predetermined watermarking method. The second and subsequent embedding are per- 
formed by using the watermarking method indicated by the recording watermarking method information included in the 

20 first i nf ormati on I wm 1 . 

[0434] Since the first embedding of the second information Iwm2 is performed by the predetermined watermarking 
method, when the second embedded information Iwm2 is extracted from the output audio data, the embedded informa- 
tion Iwm2 at the head of the output audio data can be extracted by using the predetermined watermarking method. Fur- 
ther, based on the second information Iwm2 so extracted, the information embedded in other regions o^ the output 
25 audio data can be extracted or removed. h j » ; . 

[0435] The output audio data so created is supplied to the recording unit 166. ' ' ! i; v . 

(recording of output audio data) 

30 [0436] In the recording unit 166, the output audio data supplied from the insertion unit 164 is storey in, the DVO- ir , c , 

RAM disk according to a control signal from the control unit 167. The output audio data is sequentially stij Sftf^ani : ; l^f- 

head of the recording area of the DVD-RAM disk. -}J / ■ " " "V, " J 

[0437] As described above, according to the sixth embodiment of the present invention, the data processing appa- 
ratus 160 includes, instead of the data obtaining unit 1 1 1 of the first embodiment, the data obtaining unit 161 which has 

35 the antenna for receiving a broadcast wave from a broadcast station and the decoder for decoding the received broad- 
cast wave Bw, and obtains distributed audio data as the output from the decoder. Then, apparatus 1 60 performs extrac- 
tion and removal of embedded information (first embedded information Iwm1) from the distributed audio data, and 
processes the extracted information (first information Iwm1) to create processed information (second information 
Iwm2). Then, the apparatus 160 embeds the processed intonation (second information Iwm2) in second digital audio 

40 data (processed audio data) obtained by the removal process performed on the distributed audio data, by using a 
watermarking method different from the above-mentioned watermarking method, thereby generating third digital audio 
data (output audio data). Therefore, addition or removal of information to/from the embedded information in the distrib- 
uted audio data is possible. 

[0438] Furthermore, when the processed audio data is recorded, additional information such as an apparatus iden- 
45 tification number (e.g., CPU identification number) assigned to the apparatus which performs the recording or the date 
of the recording, is embedded in the processed audio data. Thereby, if the copyright holder has found an illegal copy of 
the digital data, the copyright holder can extract more specific information to prove the illegality from the digital data. 
[0439] Furthermore, when copy permission information or the like is recorded in the digital data by a predetermined 
watermarking method, a recording apparatus adapted to the predetermined watermarking method can pert on direct 
so copy restriction, such as to inhibit copying of the digital data, according to the contents of the copy permission informa- 
tion. In this case, however, a recording apparatus which performs copying without referring to the copy permission infor- 
mation embedded by the watermarking method, cannot restrict illegal copying. 

[0440] While in this sixth embodiment the distributed audio data is obtained by receiving a broadcast wave, it may 
be obtained by receiving a signal, which has been transmitted through a cable (e.g., internet or telephone line), with a 
55 network card or a modem. -. — „ . . 

[0441 ] Further, in this sixth embodiment, the user of the data processing apparatus 1 60 may select some pieces of 
distributed audio data to be processed, from the plural pieces of distributed audio data obtained from the received 
broadcast wave. 
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[0442] Further, the distributed audio data supplied by the broadcast wave may be in the state where the audio data 
is multiplexed with video data. In this case, the audio data can be extracted from the multiplexed digital data such as a 
video object transmitted by the broadcast wave. 

[0443] While in this sixth embodiment, the output audio data is sequentially recorded from the head of the recording 
5 area of the DVD-RAM disk in the recording unit 1 66, the recording method of the output audio data in the recording unit 
is not restricted thereto. 

[0444] For example, the output audio data may be written in a vacant area of the disk which is detected according 
to the recording status. Alternatively, the output audio data may be written in an area of the disk specified by the user 
of the apparatus. Further, when a file system is constructed in the DVD-RAM disk, the output audio data is written in the 
10 disk according to the file name. In this case, since the output audio data is recorded through the file system, the output 
audio data may be written in any position in the recording area of the disk. 

Embodiment 7. 

is [0445] Figure 1 7 is a block diagram for explaining a data processing apparatus 1 70 according to a seventh embod- 
iment of the present invention. 

[0446] This data processing apparatus 170 includes a removal unit 173 for removing embedded information from 
distributed audio data, in addition to the constituents of the data processing apparatus 120 according to the second 
embodiment. 

20 [0447] The data processing apparatus 170 of this seventh embodiment includes a data obtaining unit 171 which 
obtains, from a network signal Sn on a network N, information-embedded digital audio data (distributed audio data) Dau 
as input digital audio data, and audio data attribute information which is transmitted together with the distributed audio 
data Dau; an extraction unit 1 72 which extracts the embedded information from the digital audio data Data according to 
a control signal; and a removal unit 173 which removes the embedded information from the digital audio data Dau 

25 according to a control signal. 

[0448] The data obtaining unit 171 comprises a network interface 171a which exchanges the network signal Sn 
with the network N, and a received signal decoder 171b which decodes the network signal Sn supplied from the inter- 
face 171a to generate the information-embedded digital audio data Dau and the audio data attribute information. Fur- 
ther, the extraction unit 172 comprises a method 1 extractor 172a1, a method 2 extractor 172a2, a method n 

30 extractor 172an corresponding to the above-described watermarking methods (1), (2), .... (n), respectively. In the 
extraction unit 172, the embedded information is extracted from the information-embedded digital audio data Dau by a 
desired extractor in accordance with a control signal. These method extractors are identical to those described for the 
second embodiment. 

[0449] Further, the removal unit 173 comprises a method 1 remover 173a1, a method 2 remover 173a2, .... a 
35 method n remover 173an which corresponds to the above-described watermarking methods (1), (2) (n), respec- 
tively. In the removal unit 173, the embedded information is removed from the information-embedded digital audio data 
Dau by a desired remover according to a control signal. 

[0450] For example, the method 1 remover 173a1 removes the embedded information from the digital audio data 
Dau by the watermarking method (1). The method 2 remover 1 73a2 removes the embedded information from the digital 
40 audio data Dau by the watermarking method (2). The method n remover 1 73an removes the embedded intonation from 
the digital audio audio data Dau by the watermarking method (n). 

[0451] Further, the data processing apparatus 170 includes an embedding unit 174 which embeds the extracted 
information again in the intonation-removed digital data by using a predetermined watermarking method employed in 
reproduction apparatuses as domestic electrical equipment, thereby generating output digital data; and a recording unit 

45 1 76 which stores the output digital data. 

[0452] Further, the data processing apparatus 170 includes a reproduction unit 175 which reproduces the informa- 
tion-removed digital audio data from the removal unit 173, according to a control signal; and a control unit 177 which 
controls the respective units 171-176 by using the corresponding control signals. The reproduction unit 175 comprises 
an audio decoder 1 75a which converts the intonation-embedded digital data Dau to an audio signal, and a speaker 

so 1 75b which converts the audio signal to sound. Further, the control unit 1 77 comprises a CPU which performs various 
kinds of arithmetic processing and data processing, and a RAM (Random Access Memory) which stores the operation 
program of the CPU, thereby controlling the operations of the respective units 171-176. The RAM also serves as an 
execution memory for temporarily storing the audio data or the audio data attribute information. 
[0453] To be specific, the control unit 1 77 decides the value of each method use status information In accordance 

55 with the used watermarking method information which is included in the audio data attribute information obtained by the 
data obtaining unit 1 71 , and controls the process of extracting or removing the embedded information by the method 
extractor or the method remover corresponding to each watermarking method, in accordance with the result of the deci- 
sion. Further, the control unit 177 controls the process of reproducing the audio data by the reproduction unit 177, in 
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accordance with the reproduction permission information included in the extracted information. To be specific, when the 
values of the reproduction permission information included in all of the method use status information constituting the 
used watermarking method information are "1 the control unit 1 77 permits the reproduction unit 1 75 to reproduce the 
audio data, and controls the reproduction unit 1 75 so that it reproduces the audio data. 
5 [0454] Further, in the data processing apparatus 1 70, the respective units are connected with each other by a data 
bus Dbus, and the control unit 177 controls not only the extraction unit 172, the removal unit 173, and the reproduction 
unit 1 75 but also the data obtaining unit 1 71 , the insertion unit 174, and the recording unit 1 76. 

[0455] Further, the extraction unit 1 72, the insertion unit 1 74, and the removal unit 1 73 are implemented by custom 
LSIs, respectively. The recording unit 176 is implemented by a DVD-RAM drive, wherein the output digital audio data is 
10 recorded in a DVD-RAM disk. 

[0456] Hereinafter, a description will be given of the operation of the data processing apparatus 1 70. 

(acquisition of audio data Dau and its attribute intonation Dpr) 

75 [0457] When a network signal Sn supplied from the network N through the network cable is received by the network 
interface 171a of the data obtaining unit 171 , the received network signal Sn is output to the received signal decoder 
1 71 b. The signal decoder 171 b decodes the network signal Sn to generate the information-embedded digital audio data 
Dau and the audio data attribute information Dpr which follows the audio data Dau. 

[0458] The audio data Dau so obtained is sent to the extraction unit 1 72 and the removal unit 1 73 through the data 
20 bus Dbus, while the audio data attribute intonation Dpr is sent to the control unit 1 77 through the data bus Dbus. In the 
control unit 1 77, the audio data attribute information Dpr is temporarily stored in the RAM serving as an execution mem- 
ory. 

(extraction of embedded information) 

25 

[0459] The control unit 177 controls the extraction unit 1 72 by a control signal, in accordance with the used water- 
marking method information included in the audio data attribute information Dpr. That is, according to the value of each 
of the method use status information Included in the used watermarking method information, the control unit 177 out- 
puts a control signal to the corresponding method extractor. 

30 [0460] To be specific, as described for the second embodiment, In the control unit 177, the process of supplying a 
control signal to the corresponding method extractor according to the value of the method use status data is performed 
for each of the plural pieces of method use status information (method (1) use status information ~ method (n) use sta- 
tus information) which may be used at the distribution end, and accordingly, each method extractor extracts the embed- 
ded information from the input audio data Dau in response to the control signal from the control unit 177. Thereby, plural 

35 pieces of embedded information are extracted. 

(removal of embedded information) 

[0461] The control unit 177 controls the removal unit 173 by a control signal, according to the used watermarking 
40 method information included In the audio data attribute information Dpr. That is, according to the value of each method 
use status information included in the used watermarking method information, the control unit 1 77 outputs a control sig- 
nal to the corresponding method remover. 

[0462] To be specific, in the control unit 177, initially, the value of the method 1 use status information in the used 
watermarking method information is referred to. When the value of the method 1 use status information is "1", the con- 

45 trol unit 1 77 outputs a control signal which instructs the method 1 remover 1 73a1 in the removal unit 1 73 to remove the 
embedded information from the input audio data Dau by using the watermarking method (1). Thereby, in the method 1 
remover 173a1 , the information which has been embedded by the watermarking method (1) is removed from the audio 
data Dau by the watermarking method (1). On the other hand, when the value of the method 1 use status data is "0 W , 
the control unit 1 77 does not output a control signal which instructs removal of the embedded information from the input 

so audio data Dau, to the method 1 remover 1 73a1 . 

[0463] Next, in the control unit 177, the value of the method 2 use status information , in the used watermarking 
method information is referred to. When the value of the method 2 use status information is "1 the control unit 1 77 out- 
puts a control signal which instructs the method 2 remover 1 73a2 in the removal unit 1 73 to remove the embedded infor- 
mation from the input audio data Dau by using the watermarking method (2). Thereby, in the method 2 remover 1 73a2, 

55 the information which has been embedded by the watermarking method (2) is removed from the audio data Dau by the 
watermarking method (2). On the other hand, when the value of the method 2 use status data is "0", the control unit 177 
does not output a control signal which Instructs removal of the embedded information from the input audio data Dau, to 
the method 1 remover 173a2. 
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[0464] In this way, in the control unit 177, the process of supplying a control signal to the corresponding method 
remover according to the value of the method use status data is performed for each of the plural pieces of method use 
status information (method 1 use status information - method n use status information) which may be used at the dis- 
tribution end, and accordingly, each method remover removes the embedded information from the input audio data Dau 
in response to the control signal from the control unit 177. 



(reproduction of audio data Dau) 

[0465] Thereafter, the control unit 1 77 decides the values of the reproduction permission information included in all 
w of the information obtained by the above-described information extraction process. According to the result of the deci- 
sion, the control unit 1 77 outputs a control signal indicating whether the input digital audio data Dau is to be reproduced 
or not, to the reproduction unit 175. 

[0466] To be specific, when the values of the reproduction permission information included in all of the method use 
status information constituting the used watermarking method information are "1", the control unit 177 permits the 
15 reproduction unit 1 75 to reproduce the audio data Dau. Then, under control of the control unit 1 77, the audio data Dau 
is transmitted from the removal unit 1 73 through the data bus Dbus to the reproduction unit 1 75. In the reproduction unit 
1 75, the received audio data Dau is decoded by the audio decoder 1 75a to be converted to an audio signal, and sound 
corresponding to this audio signal is output from the speaker 175b. 



20 (re-embedding of information) 



[0467] Further, in the data processing apparatus 170, independently of the process of reproducing the audio data 
Dau from the removal unit 1 73, the information extracted from the audio data Dau is again embedded in the audio data 
Dau from the removal unit 173 by a predetermined watermarking method. The predetermined watermarking method is 

25 a watermarking method employed in reproduction apparatuses as domestic electrical equipment. 

[0468] In the control unit 177, as described above, the watermarking method which has been used for embedding 
information in the target audio data at the distribution end is decided according to the value of each method use status 
information included in the used watermarking method information, and according to the result of this decision, the 
information embedded by the corresponding watermarking method is supplied to the insertion unit 1 74. In the insertion 

30 unit 124, the information supplied from the control unit 177 is again embedded in the audio data supplied from the 
removal unit 173, by the predetermined watermarking method, thereby generating output digital audio data. 



(recording of output digital audio data) 



35 [0469] The output digital audio data generated in the insertion unit 174 is supplied to the recording unit 176 and 
recorded in a recording medium according to a control signal from the control unit 1 77. The recording unit 176 is imple- 
mented by a DVD-RAM drive, and the output digital audio data is stored in a DVD-RAM disk. 

[0470] In this seventh embodiment, the control unit 177 permits the reproduction unit 175 to reproduce the audio 
data Dau only when the values of the reproduction permission information included in all of the method use status infor- 
40 mation constituting the used watermarking method information are "1 However, the condition under which the control 
unit 177 permits the reproduction unit 175 to reproduce the audio data Dau is not restricted thereto. 
[0471] For example, reproduction of the audio data Dau may be permitted when the value of at least one piece of 
reproduction permission information included in the method use status information is "1". 

[0472] Alternatively, permission for reproduction of the audio data Dau may be decided as follows. That is, a weight- 
45 ing factor is set for each of the method use status information constituting the used watermarking method information, 
and the product of this weighting factor and the value of the reproduction permission information included in each 
method use status information is obtained, and then the products obtained for all of the method use status information 
are summed up. According to whether this sum exceeds a predetermined threshold or not, reproduction of the audio 
data Dau is permitted. 

so [0473] As described above, according to the seventh embodiment of the present invention, the data processing 
apparatus 170 is provided with the data obtaining unit 171 which receives a network signal from the network N and 
obtains the information-embedded digital audio data Dau and the audio data attribute information Dpr. Extraction and 
removal of the embedded information from the audio data Dau are performed according to the method use status infor- 
mation included as the used watermarking method information in the audio data attribute information Dpr, and repro- 

55 duction of processed audio data is controlled in accordance with each embedded information. Therefore, in the data 
processing apparatus 1 70 receiving the audio data Dau, reproduction of the audio data Dau can be correctly controlled 
according to the embedded information which has been embedded in the audio data Dau by various watermarking 
methods at the distribution end. 
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[0474] Furthermore, the embedded information in the distributed audio data is again embedded in the processed 
audio data obtained by removing the embedded information from the distributed audio data, by using a predetermined 
watermarking method. Therefore, even in a reproduction apparatus as a domestic equipment which employs the pre- 
determined watermarking method, reproduction of the distributed audio data recorded on the recording medium can be 
5 controlled according to the embedded information. Furthermore, the quantity of the embedded information in the dis- 
tributed audio data recorded on the recording medium can be reduced as compared with that of the second embodi- 
ment, whereby degradation of sound quality due to the embedded information can be reduced when reproducing the 
distributed audio data recorded on the recording medium. 

[0475] While in this seventh embodiment the reproduction unit 1 75 reproduces the processed audio data, it may 
10 reproduces the output audio data which is obtained by embedding the information (embedded information) extracted by 
the extraction unit 172 again in the processed audio data. 

[0476] In this seventh embodiment, the information-embedded digital data may be obtained by embedding plural 
pieces of relevant information in the supply digital data by using a plurality of watermarking methods, and the insertion 
unit may embed a predetermined number of relevant information amongst the plural pieces of relevant information, in 
15 the input digital data, by using the corresponding watermarking methods amongst the plural watermarking methods or 
using a watermarking method which is different from these watermarking methods, thereby generating the output digital 
data. 

APPLICABLE FIELD !N INDUSTORY 

20 

[0477] The present invention relates to data processing apparatuses and data recording media and, more particu- 
larly, to a process of embedding a watermark (information to be embedded) in digital data such as audio data obtained 
by electronic distribution or from recording media, by using a specific watermarking method such as a predetermined 
watermarking method which is employed for versatile data reproduction apparatuses. 
25 [0478] Particularly, a data processing apparatus according to the present invention converts various kinds of infor- 
mation-embedded digital data corresponding to different watermarking methods, which are obtained by embedding rel- 
evant information in digital data to be supplied from the distributor to the user, into those corresponding to a versatile 
watermarking method which is employed for reproduction apparatuses which are domestic electric equipment, and this 
data processing apparatus is very useful for protecting copyrights of digital data by the watermarking method. 

30 

Claims 

1 . A data processing apparatus for receiving various kinds of information-embedded digital data which correspond to 
different watermarking methods and have been obtained by embedding relevant information relating to supply dig- 

35 'rtal data in the supply digital data to be supplied from the distribution end to the user, and processing these infor- 
mation-embedded digital data, said apparatus comprising: 

data obtaining means for obtaining a desired information-embedded digital data as input digital data; 
method information obtaining means for obtaining used method intonation which corresponds to the input dig- 

40 ital data and indicates the watermarking method used for the process of embedding the relevant information; 

information extraction means for extracting the relevant information from the input digital data by using the used 
watermarking method, on the basis of the used method intonation which has been obtained; and 
information embedding means for embedding the extracted relevant information or processed information 
obtained by processing the relevant information, in the input digital data or in processed digital data obtained 

45 by subjecting the input digital data to a predetermined data processing, by using a predetermined watermark- 

ing method. 

2. A data processing apparatus as described in Claim 1 wherein said relevant information embedded in the informa- 
tion-embedded digital data is information relating to the copyright holder of the supply digital data. 

so 

3. A data processing apparatus as described in Claim 1 wherein said information embedding means embeds the rel- 
evant information or the processed information in the input digital data by using the predetermined watermarking 
method, thereby generating output digital data. 

55 4. A data processing apparatus as described in Claim 3 further comprising a data reproduction unit for reproducing 
the output digital data. 

5. A data processing apparatus as described in Claim 3 further comprising a data recording unit for recording the out- 
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6. A data processing apparatus as described in Claim 1 further comprising: 

5 information removal means for removing the relevant information from the input digital data on the basis of the 

used method information obtained by the method information obtaining means, thereby generating informa- 
tion-removed digital data as the processed digital data; and 

said intonation embedding means embedding the relevant intonation or the processed intonation in the intona- 
tion-removed digital data by using the predetermined watermarking method, thereby generating output digital 
10 data. 

7. A data processing apparatus as described in Claim 6 further comprising a data reproduction unit for reproducing 
the output digital data. 

75 8. A data processing apparatus as described in Claim 6 further comprising a data recording unit for recording the out- 
put digital data. 

9. A data processing apparatus as described in Claim 1 wherein: 

20 said intonation-embedded digital data comprises a first data portion having a fixed length and a second data 

portion having a variable length; 

the used method intonation indicating the used watermarking method is embedded in the first data portion by 
the used watermarking method; and 

said method information obtaining means obtains the used method information indicating the used watermark- 
25 ing method, from the first data portion of the input digital data 

10. A data processing apparatus as described in Claim 1 wherein: 

said information-embedded digital data has a data structure including a plurality of data packets; 
30 the used method information indicating the used watermarking method is included in a header of a predeter- 

mined data packet amongst the plural data packets; and 

said method information obtaining means obtains the used method information indicating the used watermark- 
ing method from the header of the predetermined data packet of the information-embedded digital data. 

35 11. A data processing apparatus as described in Claim 1 wherein: 

said information-embedded digital data is followed by an auxiliary data portion; 

the used method information indicating the used watermarking method is included in the auxiliary data portion; 
and 

40 said method information obtaining means obtains the used method information indicating the used watermark- 

ing method from the auxiliary data portion which follows the information-embedded digital data. 

12. A data processing apparatus as described in Claim 1 wherein: 

45 said data obtaining means is able to obtain the information-embedded digital data from at least one of plural 

data recording media and plural data transmission paths; and 

said method information obtaining unit decides the used watermarking method in accordance with any of the 
data recording media and the data transmission paths from which the information-embedded digital data has 
been obtained, and obtains the used method information corresponding to the result of the decision. 

50 

'"T T3." A'data'pfdcessin'g apparatus as describedin Claim 1 wherein said information-embedded digital data is audio dig- 
ital data, or video digital data, or multiplexed digital data obtained by multiplexing audio digital data and video digital 
data. 

55 1 4. A data processing apparatus as" described in Claim 1 wherein said data obtaining means has an antenna for receiv- 
ing a broadcast wave from a broadcasting station, and a received-wave decoder for demodulating the wave 
received by the antenna to output the information-embedded digital data. 
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15. A data processing apparatus as described in Claim 1 wherein said data obtaining means has a data reading unit 
for reading, from a bulk memory, the information-embedded digital data stored in the bulk memory. 

16. A data processing apparatus as described in Claim 15 wherein said data reading unit has an access unit for read- 
ing the information-embedded digital data from DVD-ROM, DVD-RAM, or semiconductor recording media. 

17. A data processing apparatus as described in Claim 1 wherein: 

said information-embedded digital data is obtained by embedding plural pieces of relevant information in the 
supply digital data by using plural watermarking methods; 

plural pieces of used method information corresponding to the plural watermarking methods are included in the 
information-embedded digital data, as the used method information indicating the used watermarking method; 
and 

said intonation extracting means is able to extract the relevant information corresponding to each of the plural 
watermarking methods from the input digital data, on the basis of the used method information corresponding 
to the plural watermarking methods. 

18. A data processing apparatus as described in Claim 1 wherein: 

said information-embedded digital data is obtained by embedding plural pieces of relevant information in the 
supply digital data by using plural watermarking methods; and 

said information embedding means embeds a predetermined one of the plural pieces of relevant information 
in the input digital data by using the predetermined watermarking method, thereby generating output digital 
data. 

19. A data processing apparatus as described in Claim 1 wherein: 

said information-embedded digital data is obtained by embedding plural pieces of relevant information in the 
supply digital data by using plural watermarking methods; and 
v. < \ said information embedding means embeds a predetermined number of relevant information amongst the plu- 
:- . ; -Aral pieces of relevant information, in the input digital data, by using the corresponding watermarking methods 
amongst the plural watermarking methods or a watermarking method different from the plural watermarking 
methods, thereby generating output digital data. 

20. A data processing apparatus as described in Claim 6 wherein: 

said information-embedded digital data is obtained by embedding plural pieces of relevant information in the 
supply digital data by using plural watermarking methods; and 

said information embedding means embeds a predetermined one of the plural pieces of relevant information 
In the processed digital data by using the predetermined watermarking method, thereby generating output dig- 
ital data. 

21. A data processing apparatus as described in Claim 6 wherein: 

said information-embedded digital data is obtained by embedding plural pieces of relevant intonation in the 
supply digital data by using plural watermarking methods; and 

said information embedding means embeds a predetermined number of relevant information amongst the plu- 
ral pieces of relevant information, in the processed digital data, by using the corresponding watermarking 
methods amongst the plural watermarking methods or a watermarking method different from the plural water- 
marking method, thereby generating output digital data. 

22. A data recording medium in which information-embedded digital data is recorded, said information-embedded dig- 
ital data being obtained by embedding relevant information relating to supply digital data in the supply digital data 
to be supplied from the distribution end to the user, 

said information-embedded digital data having a data structure wherein: 

the information-embedded digital data is composed of a first data portion having a fixed length and a second 
data portion having a variable length; 
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used method information indicating the watermarking method used for embedding the relevant information in 
the supply digital data is embedded in the first data portion; 

the relevant information is embedded in the second data portion by the used watermarking method indicated 
by the used method information; and 
5 the relevant information can be extracted or removed from the second data portion by using an appropriate 

watermarking method on the basis of the used method information. 

23. A data recording medium as described in Claim 22 wherein the used method information is embedded in the first 
data portion of the information-embedded digital data by a predetermined watermarking method. 

w 

24. A data recording medium as described in Claim 22 wherein plural pieces of used method information indicating dif- 
ferent used watermarking methods, which have been used for embedding the relevant information in the supply 
digital data, are embedded in the first data portion of the information-embedded digital data. 

15 25. A data recording medium in which information-embedded digital data and used method information are recorded, 
said information-embedded digital data being obtained by embedding relevant information relating to supply digital 
data in the supply digital data to be supplied from the distribution end to the user, and said used method information 
indicating the watermarking method which has been used for embedding the relevant information in the supply dig- 
ital data and follows the information-embedded digital data, wherein: 

20 

said information-embedded digital data has a data structure in which the relevant information can be extracted 
or removed by using an appropriate watermarking method on the basis of the used method information. 

26. A data recording medium as described in Claim 25 wherein plural pieces of used method information indicating dif- 
25 ferent watermarking methods, which have been used for embedding the relevant information in the supply digital 
data, are recorded as the used method information. 
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Fig.7 
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Fig.17 
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Fig. 18(b) 
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Fig.19 
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Fig.21 
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Fig.22 
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Fig.23 
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